OBSERVABILITY · TRACE WATERFALL INSPECTOR
Agent observability: trace a failed dependency
Inspect actual span IDs, parent relationships, sampling decisions and a deterministic execution schedule.
Your experiment
Inject an error in the first database dependency. Inspect propagated parent errors and explain why four error spans still describe only one failed request.
Click any waterfall row or use the span selector. A gold outline marks your selection; details include the exact parent ID, timestamps, status and export decision.
The one-minute inspection is a slowed visualization of the modeled millisecond timeline, not a measured request. Inclusive parent and child durations overlap.
Selected span
Offline schedule and trace-context exercise; no agent, network request or OpenTelemetry exporter runs. Header validation deliberately supports version 00 only and is not a complete forward-compatible W3C parser. IDs are deterministic teaching fixtures, not production ID generation. Later tool durations increase by 25% of the first tool duration; database work occupies 20–80% of each tool span. A dependency failure marks its ancestors as failed in this application model. Sampling is a local policy, not an authorization boundary. Newly rooted remote traces default to sampled when context is not forwarded. The canvas shows modeled work even when telemetry is not exported, exposing the blind spot. No clock skew, spans lost in transport, baggage, tracestate or real latency is modeled. W3C Trace Context · OpenTelemetry trace concepts.
Locate a failed dependency without inventing delay
An incoming request wraps agent work, remote tool spans and database children. The failure control marks an actual branch in this explicit span tree, rather than adding an unrelated warning badge. The error is visible on that database and on the modeled ancestors that depend on it. Start by selecting individual spans and reading their parent IDs. This separates a failed leaf from the higher-level spans that report the same failure and prevents counting each ancestor as a separate user request.
Configure the schedule and context
Incoming traceparent accepts version-zero lowercase hexadecimal fields with nonzero trace and parent IDs. Invalid input starts a new synthetic root and displays a validation message. Forward context to remote tools chooses whether tools retain the incoming trace or start separate roots. Run tools in parallel changes their schedule. Remote tools selects one through four branches. Local work before and after tools supplies both local intervals. First tool duration is the base duration; each later tool adds twenty-five percent of that base per index. Dependency fault marks either the first database or every database as failed and propagates the error through modeled ancestors.
Reproduce one root cause and four error spans
Keep three tools, forty milliseconds of local work and a two-hundred-millisecond first tool. With parallel execution and no fault, request duration is 390 milliseconds. Select First database fails: Database 1, Tool 1, Agent work and Incoming request become error spans, four in total. Duration stays 390 milliseconds because this fixture changes outcome, not service time. Switch to all databases failing and all eight spans become erroneous, while there is still only one modeled incoming request. Disable capture to see the same failure path without any exported spans.
Inspect recording independently of execution
Capture and export spans switches recording on or off without removing the modeled execution. Honor incoming unsampled flag applies when a valid incoming header is unsampled; respecting that decision suppresses the propagated trace. New remote roots default to sampled if context forwarding is disabled. Inspect a span selects its exact JSON record, and clicking a waterfall row makes the same selection. Play and Pause reveal the schedule across sixty inspection seconds. Next span boundary jumps to a start or end; Restart rewinds; Speed changes viewing pace; Inspection time scrubs. Parameter changes restart playback unless reduced motion is requested. The full-scenario preview and selected JSON are not limited to already completed spans; the filled bars show inspected progress.
Know what the fixture does not observe
Database work occupies twenty through eighty percent of its enclosing tool duration. IDs are deterministic teaching identifiers, not production generators. The parser intentionally covers version zero rather than all forward-compatible trace-context cases. No actual OpenTelemetry collector, exporter acknowledgement, baggage, tracestate, network span loss, clock skew or ADK agent is present. An exported-span count is a configured local recording decision, not proof that a backend accepted the record. Overlapping inclusive spans must not be added to claim request latency. The synthetic schedule supports reasoning about causality, visibility and aggregation; it does not measure an installed service.