RELEASE ENGINEERING · TRAFFIC CONTROL ROOM
Canary release: compare observed cohorts
Route individual requests, observe both versions and stop candidate traffic when the measured release gate fails.
Scroll the diagram left or right to inspect every label.
Your experiment
Keep the initial share small. Increase minimum sample count and explain why a seemingly healthy canary may still have insufficient evidence.
The preview is the computed end of this same replay. Capture a baseline, change one parameter and compare the cohort evidence. “Within limits” is not proof of safety or an automatic production promotion.
Recent request trace
Offline, seeded traffic exercise; no deployment, provider API or real flag service is used. Two requests per second are evaluated without a capacity queue. Control latency is 100–200ms; candidate latency spans 75–125% of its configured center. Outcomes use independent fixed pseudo-random streams. The gate requires enough samples in both cohorts, then rejects an excessive observed error-rate difference or candidate p95. It is not a statistical significance test. Metrics accumulate from start, without a sliding window, confidence interval or dependency failures. Shadow responses never serve users; safe handling of real side effects is not implemented. Google SRE: canarying releases.
Judge observed cohorts within a declared fixture
The replay evaluates one hundred twenty requests at two requests per model second. Control and candidate use fixed independent pseudo-random streams, allowing identical settings to reproduce identical outcomes. The gate waits for the configured minimum in both cohorts, then compares observed error-rate difference and candidate p95 latency. These are actual calculations over the generated rows, but the probabilities and latencies are synthetic inputs. Within configured limits is not a statistical significance result or automatic production approval.
Choose routing and incident inputs
Strategy selects constant-share canary, progressive exposure, blue/green cutover or shadow copies. Initial candidate share applies to canary and progressive; progressive adds twenty-five percentage points each fifteen seconds up to one hundred. Blue/green cutover chooses its switch instant. Shadow keeps control user responses and sends copies to the candidate. Control error probability governs control and the candidate before the fault begins. Candidate error probability takes effect at Candidate fault begins. Those probabilities are inputs, not the finite cohorts’ exact observed percentages.
Set evidence and rejection rules
Minimum samples per cohort prevents decisions before both sides have enough observations. Allowed error difference is a percentage-point gap, not a relative percent increase. Candidate latency center sets the distribution from seventy-five to one hundred twenty-five percent of that center. Candidate p95 limit is a hard ceiling on the nearest-rank observed percentile. Stop candidate traffic on rejection makes future candidate share zero after a failing gate; it does not undo prior requests. Without it, a rejection remains visible while traffic continues.
Reproduce an insufficient-control boundary
Select canary and one hundred percent candidate share. At completion no control samples exist, so the gate stays Insufficient samples even if many candidate outcomes were observed. Reduce share to twenty-five percent, keep minimum eight, and inject seventy-percent candidate errors from time zero. With error allowance zero and automatic stopping enabled, the gate rejects once both cohorts have evidence and later candidate traffic stops. In shadow mode, candidate errors can fail the gate while user errors still come only from control responses.
Follow requests rather than a promotion slogan
Play and Pause reveal the sixty-second trace; Next request advances half a model second, Restart rewinds, Speed changes pace and Time scrubs. Input edits recompute and rewind. Metrics accumulate from the start and are not sliding windows. There are no confidence intervals, session pinning, request capacity queues, correlated failures or actual deployment actions. Shadow side effects are not implemented, so safe shadowing of mutating production calls requires a separate design. Compare cohort sizes, denominators and rollback events before interpreting a final label.