OpenTelemetry Demo restores ad metrics and checks collector overlays

OpenTelemetry Demo, a distributed application for demonstrating telemetry, fixed a configuration bug that silently dropped ad service metrics. The October 15 to 21 activity also added a check for collector overlays and metrics for the agent, MCP, and chatbot services. These changes matter if you use the demo to test telemetry pipelines or copy its configuration into a fork.

Ad metrics survive the full collector configuration

The receiver fix in PR #4122 adds prometheus/ad to the metrics pipeline in otelcol-config-full.yml. The base configuration already defined and used that receiver. The full layer omitted it, so the ad service’s Prometheus endpoint was never scraped in layered configurations, including the default make start.

The underlying rule is easy to miss. When the collector merges multiple --config files, arrays replace earlier arrays. Listing extra receivers in a later file therefore requires repeating the existing receivers too. Here, the missing entry prevented demo_ad_served_total from reaching the pipeline. A receiver definition alone did not keep the scrape active.

The same fix updates the customization example in otelcol-config-extras.yml. Its commented receiver list now also includes postgresql, prometheus/ad, and kafkametrics. If you copied the older example into a fork, compare your active list with the upstream pipeline before adding another exporter.

CI catches components lost between layers

PR #4148 adds a pipeline preservation check. It reads the base, full, and observability configurations in their load order. For each pipeline, it checks whether a later receivers, processors, or exporters list drops an entry from an earlier layer.

The implementation commit adds check_collector_pipelines.py and runs it in CI. A failure reports the layer, pipeline, component list, and missing entries, then exits with status 1. That gives contributors a specific configuration error to fix before the change merges.

There is a deliberate boundary. The check excludes the extras file because forks may replace lists there intentionally. It also checks membership, so it does not establish that processor ordering is correct. Treat it as protection against accidental omissions when editing the standard layers.

Agent services gain counters and a Grafana dashboard

The agent metrics change adds demo.tool.calls for the agent and MCP services, plus demo.chatbot.requests for messages sent through the chatbot UI. Tool calls carry gen_ai.tool.name, allowing queries to separate operations such as list_products and add_to_cart.

The counters increment before the tool or request completes. Read them as invocation volume, not successful operation counts. That distinction matters when a tool fails but its counter continues rising.

An “Agent Services” Grafana dashboard brings these counters together with LLM call duration and token metrics from the existing instrumentation. The change also registers the new metrics in the telemetry schema and marks metrics as expected for all three services in the telemetry test matrix. Contributors now have explicit coverage expectations when changing these services.

What to watch

• Check for demo_ad_served_total after updating a layered demo deployment. If it remains absent, inspect any custom receiver list that loads after the standard configurations.
• Review Helm configuration separately. PR #4122 carries the helm-update-required label; that label does not establish that your installed chart includes the fix.
• Keep custom extras lists under review. The new CI check intentionally leaves them outside its scope, so a passing check cannot confirm that a fork preserves every upstream component.