Span vs Trace: Understanding the Building Blocks of Distributed Tracing

A span is a single unit of work — one function call, one database query, one HTTP request. A trace is the full tree (or DAG) of spans that represent one complete request.

Said differently: a trace is made up of spans. One trace = one request through your system; many spans = many operations that happened during that request.

The analogy: a trace is like a book, and spans are the chapters. The book is the whole story; chapters are the discrete sections that make it up.

A span represents a single operation: an HTTP request, a database query, a function call, a message publish, a cache lookup. It has a defined start and end.

Every span has: a name (operation description), a start timestamp, a duration, attributes (key-value metadata like http.method = "GET"), events (timestamped log-like entries), and status (ok, error).

Spans have identifiers: a span_id (unique to this span) and a trace_id (shared across all spans in the same trace). Child spans record their parent_span_id, which is how the tree is reconstructed.

Example span attributes: http.method = "POST", http.url = "/api/orders", http.status_code = 500, user.id = "42", db.statement = "SELECT * FROM orders WHERE id = $1".

A trace is the complete set of spans that resulted from a single root request — typically a user-initiated action or an inbound HTTP call.

The root span is the one with no parent. All other spans in the trace have a parent_span_id pointing to another span in the same trace.

A simple REST request might produce 3 spans (incoming HTTP, database query, response). A complex microservices request can produce 50+ spans spanning 10+ services.

Traces are visualized as flame graphs: time runs left-to-right; spans are stacked vertically. The widest spans are the slowest; the deepest spans are the most nested.

A user clicks "Checkout" on an e-commerce site. The resulting trace has five spans:

1. Root span: POST /checkout (web service, 320ms total).

2. Child of (1): GET /cart (cart service, 15ms).

3. Child of (1): POST /charge (payment service, 180ms).

4. Child of (3): external call to Stripe API (160ms — most of (3)).

5. Child of (1): INSERT INTO orders (database query, 25ms).

All five spans share the same trace_id. Each span has its own span_id. Spans 2, 3, and 5 have parent_span_id = span 1. Span 4 has parent_span_id = span 3.

For a span in Service B to know it is a child of a span in Service A, Service A must tell Service B the trace_id and parent span_id when calling it.

For HTTP, this is done via headers. The W3C Trace Context standard defines traceparent and tracestate headers that every HTTP library should pass through.

For async messaging (Kafka, RabbitMQ, SQS), context travels in message headers. OpenTelemetry propagators abstract this across protocols.

Without context propagation, each service produces its own disconnected trace — you lose the end-to-end view.

Think in terms of traces when debugging a user-facing issue: "Why was this /checkout request slow?" You open the trace, see the flame graph, identify the slow span.

Think in terms of spans when instrumenting new code: "This domain function should be a span so I can see its timing in traces". Add a custom span around any operation whose timing you want to measure.

Aggregate span-level analytics for operational insight: "Average db.query span duration for /api/orders, bucketed by hour".

Modern APM tools like Atatus make both easy — one UI shows individual trace flame graphs and aggregate span analytics with the same underlying data.