Skip to main content

Traces

Overview​

The Traces page in CtrlB is used for distributed tracing to visualize the lifecycle of a request as it passes through a set of applications.

  • A trace represents the complete journey of a request across services.
  • A span is an individual operation within that trace.

Tracing provides deep insights into your system by identifying:

  • Performance bottlenecks – pinpoint where requests spend the most time.
  • Errors – detect failed spans and trace their root causes.
  • Service dependencies – understand how microservices interact within a request.

Traces offer a visual timeline of the request flow, showing how long each step takes, where additional requests begin and end, and how the system responds overall. This makes it possible to uncover hidden problem areas that would be difficult to identify without end-to-end tracing.

  1. From the sidebar, click Traces.
  2. Use the time range selector (e.g., Last 12 hours) to choose the analysis window.
  3. Optionally load a Saved View for recurring investigations.
  4. Run queries to filter traces by service, operation, or status.

Traces Dashboard
Main Traces page

Traces Table​

The default table view lists all spans within the selected timeframe.

ColumnDescription
Start TimeTimestamp when the span started.
Service NameThe microservice executing the span (e.g., payment-service).
Operation NameThe specific operation or function executed (e.g., process-payment).
StatusExecution status: OK, ERROR, or UNSET.
DurationTime taken to complete the span (ms/µs).
Span IDUnique identifier of the span.
Trace IDIdentifier for the entire trace (shared by all spans in a request).

Traces Table
Filtered spans

We can either use the attributes filter panel on the left or search directly from the search-bar to narrow down results:

1. Filter Panel​

1. Span Status​

Filter spans by execution result:

  • OK – successful spans.
  • ERROR – failed spans.
  • UNSET – spans with no status reported.

2. Span Duration​

  • Define Min and Max Duration (in microseconds).
  • Useful to isolate long-running spans.

3. Attribute Filtering​

Expand sections like General, HTTP, or Code.
Common attributes include:

  • http.method – request type (GET, POST).
  • http.host – host or domain.
  • code.filepath – source code file.
  • code.function – function name.

Filter Panel
Filtering by status, duration, and attributes.

Use the search-bar to filter and search spans with desired attributes :

  • Add fields like _timestamp, duration, end_time, code_function, or http.method
  • Focus only on selecting attributes or data points relevant to your investigation.

Attribute Selector
Selecting attributes to filter the spans

Detailed Trace View​

Click a Trace ID to open its details.

1. Header​

  • Trace ID – unique identifier for the request.
  • Root Service – initiating service.
  • Root Operation – entry-point operation.
  • Duration – total request time.
  • Spans – number of spans in the trace.

Trace Header

2. Visualization Modes - Span Specific Details​

When you click on spans - the details of the related trace will be shown in Waterfall and Table format.

a. Waterfall​

  • Timeline view of spans (Gantt-style).
  • Shows parent-child relationships.
  • Ideal for spotting latency bottlenecks.

Waterfall View

b. Table​

  • Tabular span listing with raw metadata.
  • Includes fields like _timestamp, duration, end_time, events, flags, and links.

Trace Table

Service Map​

  • Dependency visual of service interactions.
  • Helps to understand cross-service flows or how spans travelled through microservices.

3. Span Info Panel​

Clicking a span opens its metadata in the right-hand panel.

Key fields include:

  • operation_name – executed function.
  • service.name – service name.
  • span_id – span identifier.
  • trace_id – trace identifier.
  • reference_parent_span_id – parent span ID.
  • span_status – execution result.
  • duration – time taken.
  • start_time / end_time – precise timestamps.
  • events, flags, links – advanced debugging metadata.

Span Info Panel
Span Info Panel

Example Scenarios​

  • Payment delays: payment-service → process-payment spans take 300ms while others complete in 1ms → investigate payment system.
  • User onboarding failure: Root operation create-user shows ERROR spans → check downstream services.
  • Deployment validation: Compare trace durations before and after release.

Best Practices​

  • Instrument services with OTel, Fluent Bit, or any supported collector.
  • Use consistent service and operation names for clarity.
  • Apply retention policies to manage data volume.
  • Combine traces with alerts for proactive monitoring.
  • Configure sampling to reduce overhead in high-throughput systems.