All 25 articles, sorted alphabetically
Grafana
Data source federation. Loki + Tempo. Grafana Cloud + Alloy.
Read article →Loki vs Elastic vs ClickHouse for Logs
Cost, query speed, and the cardinality story.
Read article →Adaptive trace sampling architecture
Deep-dive on adaptive sampling for distributed tracing: why fixed-rate sampling fails, a rate governor that measures per-operation throughput and allo…
Read article →Metrics anomaly detection architecture
Deep-dive on anomaly detection for metrics at scale: series routing and windowing, seasonal baseline stores, cheap detector ensembles with persistence…
Read article →Burn-rate SLO alerting architecture
Deep-dive on multi-window multi-burn-rate SLO alerting: why a burn rate (observed error ratio divided by the budget the SLO permits) turns an alert in…
Read article →Metric cardinality -- the silent cost driver of observability
Deep-dive on metric cardinality: every label combination as a distinct time series, cardinality as the product of label value counts, explosions from …
Read article →Dashboards-as-code architecture
Deep-dive on building observability dashboards as versioned code: shared panel and query libraries for consistency, CI validation that catches broken …
Read article →eBPF Observability Architecture in Depth
A 2500-word walkthrough of eBPF observability: user app, kernel attach points, verifier + JIT, maps, network/syscall/user probes, aggregator, export.
Read article →Full-Stack Observability Architecture in Depth
A 2500-word walkthrough of a modern observability stack: OTel, collector, metrics/traces/logs/profiles/eBPF, alerting, dashboards, and on-call.
Read article →Head-based trace sampling architecture
Deep-dive on head-based sampling for distributed tracing: the root-span sampling decision, the sampled flag in propagated trace context, consistent tr…
Read article →Health check architecture
Deep-dive on health checking: why liveness, readiness, and startup probes ask different questions, shallow vs deep checks, how a shared /health endpoi…
Read article →Latency histograms and quantile estimation architecture
Deep-dive on latency histograms: hot-path bucket counters, why percentiles don't average, cumulative buckets and mergeability…
Read article →Metric downsampling architecture
Deep-dive on metric downsampling: why lossy aggregation must preserve the extremes incidents live in, the tiered storage model and cascading rollups, …
Read article →Metrics
How metrics work: counters, gauges, histograms, and dimensional cardinality.
Read article →OpenTelemetry Collector architecture
Deep-dive on the OTel Collector: receiver/processor/exporter pipelines and connectors, daemonset agents vs gateway fleets, trace-affinity tail samplin…
Read article →Recording rules architecture
Deep-dive on monitoring recording rules: evaluating expensive queries on a fixed interval and storing results as cheap derived series, the rule evalua…
Read article →Real User Monitoring
Deep-dive on Real User Monitoring: browser SDK instrumentation, Core Web Vitals (LCP/INP/CLS), session and error tracking, beacon transport, field vs …
Read article →OpenTelemetry semantic conventions architecture
Deep-dive on OpenTelemetry semantic conventions, the standardized registry of attribute names, types, and values that makes telemetry correlatable acr…
Read article →Span metrics architecture
Deep-dive on span metrics: the OTel Collector spanmetrics connector, RED metrics (rate/errors/duration) with dimensions, exemplars linking metrics to …
Read article →Synthetic monitoring -- testing from the outside, always
Deep-dive on synthetic monitoring: proactive scripted probes (uptime, API, browser flows), running from many locations, the comparison to real user mo…
Read article →Tail-based trace sampling architecture
Deep-dive on tail-based sampling: why head sampling loses incident evidence, trace-ID-aware load balancing, decision buffers and windows, keep policie…
Read article →Trace context propagation architecture
Deep-dive on distributed trace context propagation: the W3C traceparent wire format, inject/extract at every boundary, parent/child span linkage, bagg…
Read article →Distributed Traces
How distributed traces link operations across services via trace context propagation.
Read article →On-Call Runbooks
Every alert has a runbook. Live document. Actual practice.
Read article →Prometheus
Pull-based metrics. TSDB. PromQL. Alerting.
Read article →