What is OpenTelemetry?

The problem it solves

In a typical company, system metrics, application logs, infrastructure logs, security data and business metrics end up in different tools, each with its own agent, its own format and its own protocol. When something breaks, correlating a log line with the metric that spiked in the same minute takes three open consoles and a lot of patience. For years the market has produced proprietary and open source solutions that barely talk to each other.

OpenTelemetry was born to solve this problem. It fixes a common data model for the signals, a transport protocol, a set of naming conventions and a set of software components that apply them. The promise is to write the instrumentation once and reuse it with any analysis backend, today and three years from now. The project site sums up the promise in two principles: the data you generate is yours, with no vendor lock-in, and there is a single set of APIs and conventions to adopt.

What it does and what it does not

OpenTelemetry generates, collects, processes and ships telemetry. It stops there. It does not store data, does not query it, does not draw dashboards and does not send alerts: those functions belong to the observability backend, which can be a commercial product such as Datadog or Grafana Cloud, an open source tool such as Grafana, Elastic or ClickStack, or a cloud provider's native service.

This separation explains the broad support the project enjoys. For those adopting the standard the advantage is obvious: you can switch backend without redoing the instrumentation.

There is a flip side. The standard comes from the consensus of dozens of companies and communities, and it evolves at the pace of a committee: the specifications are broad, some parts mature more slowly than others, and the complexity is far from trivial. Adopters accept this trade-off in exchange for neutrality.

How it is built

The official documentation lists about ten components: the specification, the OTLP protocol, the semantic conventions, the APIs, the SDKs for the various languages, the instrumentation libraries, automatic instrumentation, the collector and a set of supporting tools such as the Kubernetes operator and the Helm charts. To find your way around, it helps to reduce them to three blocks.

Instrumentation is what produces telemetry inside applications and infrastructure: the libraries and agents that open traces, count requests, emit logs. SDKs exist for every language in common use, from Java to Go, from Python to .NET, from JavaScript to PHP.

The protocol, called OTLP, is the format in which telemetry travels from the application to the collector and from the collector to the backend. It is the same for all signals, works in push mode over gRPC or HTTP, encodes data as Protocol Buffers or JSON and is stateless, so it scales horizontally behind a load balancer. One detail that proves valuable in production: a destination close to saturation can ask the source to slow down.

The collector is the agent that receives, processes and exports.

Signals

OpenTelemetry calls the categories of telemetry it handles signals. The main supported signals are traces, metrics and logs.

Two mechanisms hold different signals together. Context propagation ensures that a log line and a span produced by the same request share the same trace identifier, even when they come from two different services: it is the basis of correlation, and the reason why with OpenTelemetry you go from a log line to the trace that generated it in one click. Semantic conventions fix attribute names, from service.name to http.response.status_code, so that two teams or two languages describe the same thing in the same format.

The OpenTelemetry collector

The collector is a single executable, also distributed as a container image, that sits between those who produce telemetry and those who analyse it. It is configured in YAML by declaring components and wiring them into pipelines, one per signal.

Receivers take data in: OTLP from applications, but also pre-existing formats such as Prometheus, syslog and log files. There are more than ninety of them. Processors transform data in transit: they add metadata, strip sensitive information, filter, sample. Exporters send the result to one or more destinations, about fifty of them across commercial backends, open source tools and message queues. On top of these come connectors, which join two pipelines, and extensions for authentication, health checks and local storage.

On Kubernetes the standard architecture has one collector per node as a DaemonSet, reading the pods' standard output and the kubelet metrics, and a single collector as a Deployment for cluster-level data such as events.

Where to start

In implementation projects I generally follow these steps:

1. Collect the telemetry that already exists. A collector on every virtual machine or node reads log files, system metrics, Prometheus endpoints, Windows event logs. No changes to the applications, broad coverage in a few weeks, and the ability to work on third-party software that nobody can touch. The limit: unstructured logs have to be parsed with regular expressions, and correlation between signals is weak or absent.
2. Automatic instrumentation, without touching the code. The .NET, Go, Java, JavaScript, PHP and Python SDKs can hook into the most common libraries (HTTP servers, database clients, queues) and produce traces and metrics that already follow the semantic conventions. It is enabled with an environment variable or a module loaded at startup. The limit is that you only see what the instrumented libraries see.
3. Instrumentation in the code. Developers create traces, define their own metrics and write logs through the OpenTelemetry API. This is the step that gives full correlation and the best long-term return, and it is also the most expensive because it requires changing the applications.

This approach makes sense for those who already have applications and infrastructure in production. Those starting from scratch can skip the first two steps and instrument in the code from day one.

Four practices make the journey less painful.

• Give developers direct access to the observability consoles, including production, which in environments with a lot of legacy is less obvious than it sounds.
• Build and maintain small internal wrappers around the SDKs, so that instrumenting a new service is one import line and not a page of repeated code.
• Write adapters that convert the telemetry already produced by existing systems into the new formats.
• Plan a period in which the old and the new system run side by side.

OpenTelemetry and artificial intelligence

OpenTelemetry meets LLMs in two ways.

• Observability of AI systems. A call to an agent hides a chain of model invocations, calls to external tools and token exchanges, and without telemetry you cannot tell a slow model from an agent repeating the same step dozens of times. The OpenTelemetry semantic conventions for GenAI extend OpenTelemetry to support the operation of LLM models.
• Agents in support of observability. Querying the data, finding the cause of an incident, writing a report. Here the quality of the result depends little on the model and a lot on the data it has access to. Telemetry that is well structured, correlated and queryable from a single database is what lets an agent reach the cause in a few steps, as I describe in the post I haven't opened a dashboard in six months. OpenTelemetry provides exactly that structure.

Adoption

I have been working with OpenTelemetry in client projects for a few years, and I am convinced it reached the maturity to be the default choice for logs, metrics and traces some time ago. It is not free of caveats: the specification is broad and the documentation is patchy in places, but the benefits outweigh the problems. I talked about it at two Container Day editions: in 2024 on logs in OpenTelemetry and in 2025 on incremental adoption strategies.