Skip to main content

Ingesting Logs To CtrlB via OpenTelemetry Collector

This document provides instructions for configuring the OpenTelemetry Collector to forward logs to CtrlB instances. The OpenTelemetry Collector serves as a centralized log processing and forwarding component.

Required Configuration Parameters​

  • Instance Host: <INSTANCE_HOST>
  • API Authentication Token: <API_TOKEN>
  • Stream Identifier: <STREAM_NAME>
  • Otel Collector VM IP: <COLLECTOR_IP>

Protocol Specifications​

The CtrlB platform supports OpenTelemetry Protocol (OTLP) log ingestion with the following specifications:

  • Ingestion Protocol: OTLP over HTTP (recommended) or gRPC
  • Authentication Method: HTTP Basic Authentication via Authorization: Basic <API_TOKEN>
  • Stream Routing: The stream name is part of the endpoint path (/engine/api/default/<STREAM_NAME>/_otel). The collector appends /v1/logs.
  • Timestamp Processing: Platform automatically derives _timestamp from OTLP event timestamps

Basic Configuration​

Create a configuration file named collector.yaml with the following specification:

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318 # OTLP HTTP receiver endpoint
      grpc:
        endpoint: 0.0.0.0:4317 # OTLP gRPC receiver endpoint (optional)

processors:
  batch: {}

exporters:
  otlphttp:
    endpoint: https://<INSTANCE_HOST>/engine/api/default/<STREAM_NAME>/_otel
    headers:
      Authorization: Basic <API_TOKEN>

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp]

Deployment Methods​

Container Deployment​

Deploy using Docker:

docker run --rm \
  -p 4317:4317 -p 4318:4318 \
  -v $PWD/collector.yaml:/etc/otelcol/config.yaml \
  otel/opentelemetry-collector-contrib:latest

Linux System Service​

  1. Install the OpenTelemetry Collector from here.
  2. Place configuration at /etc/otelcol/config.yaml
  3. Start service: systemctl start otelcol

Application Configuration​

HTTP Protocol Configuration​

Configure OpenTelemetry SDKs with:

export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://<COLLECTOR_IP>:4318
export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://<COLLECTOR_IP>:4318/v1/logs

gRPC Protocol Configuration​

For gRPC transport:

export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_EXPORTER_OTLP_ENDPOINT=http://<COLLECTOR_IP>:4317

Data Validation​

  1. Access your CtrlB instance dashboard
  2. Apply filters for service.name within the previous 5-minute window

Troubleshooting Guide​

Authentication Issues​

HTTP 401 Unauthorized

  • Verify the Authorization header format: Authorization: Basic <API_TOKEN>
  • Confirm API token is properly encoded

Authorization Issues​

HTTP 403 Forbidden

  • Confirm <STREAM_NAME> in the endpoint path matches a stream the token can write
  • Confirm API token has appropriate stream access

Endpoint Issues​

HTTP 404 on /v1/logs

  • Ensure the endpoint is https://<INSTANCE_HOST>/engine/api/default/<STREAM_NAME>/_otel without a trailing slash
  • The OTLP exporter appends /v1/logs itself. Including /v1/logs in endpoint produces .../_otel/v1/logs/v1/logs

Timestamp Issues​

Incorrect Event Timestamps

  • Verify system clock synchronization
  • Confirm application timestamp generation
  • Check for clock skew between components

Performance Issues​

Degraded Processing

  • Maintain batch processor configuration
  • Enable gzip compression
  • Scale Collector resources based on volume

For information about ingesting traces using the OpenTelemetry Collector, please refer to our OpenTelemetry Collector for Traces documentation.