Observability, Health Checks, and Deployment
Instrument services with logs, metrics, and traces, design meaningful health probes, publish deterministic containers, and deploy with safe configuration and rollback signals.
Before this lesson
Correlate logs metrics and traces
Design liveness and readiness probes
Plan deterministic safe deployments
The short answer
Observability connects structured logs, metrics, and distributed traces through stable context. Liveness answers whether the process should restart; readiness answers whether it should receive traffic. Deploy immutable artifacts with validated configuration and explicit rollback criteria.
Build the runtime mental model
Logs describe discrete events, metrics aggregate numeric behavior, and traces connect operations across service boundaries. OpenTelemetry provides vendor-neutral instrumentation and context propagation. None of the three replaces the others.
Advanced C# work improves when you separate language syntax, runtime behavior, and application policy. Write down which layer owns the guarantee in this lesson. Then identify the observable evidence—a compiler rejection, test result, generated query, trace, or measurement—that would prove the model correct.
Design the boundary deliberately
Health checks should be cheap and purposeful. Liveness must not fail because a downstream service is temporarily unavailable or restart storms follow. Readiness can remove an instance from traffic while essential startup or dependency conditions recover.
The starter isolates one part of the mental model so it can run in the browser. The exercise moves the same rule into a current local .NET project where packages, framework hosting, diagnostics, and multi-file tests are available.
using System;
using System.Diagnostics;
class Program
{
static void Main()
{
Stopwatch operation = Stopwatch.StartNew();
int processed = 3;
operation.Stop();
Console.WriteLine("processed=" + processed);
Console.WriteLine("duration_ms=" + operation.ElapsedMilliseconds);
}
}Expected output
processed=3
Diagnose failure and misuse
High-cardinality metric labels create cost and instability. Logging every request body leaks data. A health endpoint that always returns healthy is decoration, while one that performs an expensive full dependency audit can cause its own outage.
Classify each failure as a contract violation, transient operational failure, permanent dependency response, concurrency conflict, or programmer defect. That classification determines whether to reject, retry, compensate, cancel, or fail fast. A generic catch-and-continue policy destroys the information needed to make that decision.
| Question | Evidence to inspect | Decision |
|---|---|---|
| Is the input valid? | Validation result and boundary examples | Reject with a stable contract |
| Is the failure transient? | Typed status, exception, and policy context | Retry only when bounded and safe |
| Is state still consistent? | Invariant and transaction outcome | Commit, compensate, or abort |
| Is performance acceptable? | Representative latency and allocation data | Keep simple or optimize one cause |
Apply the concept in production
Build once and promote the same artifact, run as a non-root identity where possible, expose only required ports, handle SIGTERM with graceful drain, apply migrations safely, and tie progressive rollout to error, latency, and saturation signals.
Finish by making the result operable. Add structured diagnostics at the boundary, propagate cancellation, avoid sensitive data, and record SDK and dependency versions. Test the public behavior instead of private implementation details. If a framework or provider performs translation, serialization, concurrency, or I/O, include at least one test against the real production technology.
A senior-level review should be able to answer four questions: what contract is promised, who owns lifetime and cleanup, how failures become visible, and what evidence supports the design. If any answer depends on “the framework probably handles it,” inspect the documentation or runtime behavior and turn the assumption into a checked decision.
Quick knowledge check
Answer before you reveal.
01Why should liveness avoid depending on a downstream service?
A downstream outage could restart every healthy instance repeatedly, worsening the incident instead of allowing recovery.
02What must happen before adding complexity to this design?
State the requirement, preserve a correct baseline, collect evidence, and explain how the proposed mechanism improves a specific quality.
Exercise
Practice challenge
Instrument an API use case with trace context, structured logs, latency/error metrics, meaningful health probes, and a container deployment checklist.
Requirements
- The implementation states its contract and ownership boundary explicitly
- Automated checks cover the successful path and at least two meaningful failures
- Diagnostics expose failure context without secrets or swallowed exceptions
- The project documents required SDK, packages, setup, run, and test commands
Optional extension: Measure or load-test the critical path and record whether the evidence justifies another optimization or abstraction.
Open in C# compilerLesson checkpoint
One small step locks it in
Mark this lesson complete, then keep the momentum going.
Clear up the details
Frequently asked questions
When should I use observability, health checks, and deployment?
Use it when its explicit tradeoff solves a measured requirement or clarifies an owned boundary. Keep the simpler design when the additional mechanism does not improve correctness, operability, or changeability.
Does the browser compiler cover the complete production setup?
No. It runs the focused starter program. Framework, package, database, benchmark, and multi-project work requires a current local .NET SDK and the project commands described in the exercise.
What evidence should I keep after the exercise?
Keep the acceptance cases, automated tests, diagnostic or benchmark output where relevant, and a short decision note describing the chosen boundary and rejected alternative.