Distributed Tracing

A single user request to "place an order" might touch a dozen separate services - auth, inventory, payments, notifications - each with its own logs and metrics. When that request is slow, logs and metrics alone can tell you a service was slow. Tracing tells you which one, in this specific request.

Traces and spans

A trace represents one request's entire journey. It's made up of spans - one per unit of work, each with a start time, duration, and a parent:

Trace: place-order (420ms total)
├── span: auth-check        (10ms)
├── span: inventory-check   (30ms)
├── span: payment-charge    (350ms)  ← the slow one
└── span: send-confirmation (15ms)

Laid out like this, the slow span is obvious at a glance - something logs scattered across four different services would never show you nearly as directly.

Propagating the trace ID

The mechanism that stitches all those spans back into one trace: every service, when it calls the next one downstream, passes along a trace ID in the request headers. Every span generated anywhere in that call chain gets tagged with the same ID, so a tracing backend can later reassemble the full picture from spans that were emitted by entirely separate services.

Why logs and metrics alone fall short here

• A metric tells you "payment-charge's p99 latency went up." It can't tell you if this specific slow request also happened to hit a slow inventory check.
• A log from the payment service tells you about the payment service. It has no idea what happened in auth or inventory for that same request.
• A trace ties all of it to one request ID, across every service it touched.

Tools

OpenTelemetry has become the standard way applications emit trace data; Jaeger and Zipkin are two of the most common backends for storing and visualizing it.

Try it

curl -I https://api.internal/checkout

When you're ready, hit Mark complete and move to the next lesson.