Synthetic monitoring: one-region alert versus quorum

Paged by a probe host

The default scenario has two incidents: from 6 to 10 min the us-east probe host loses its network while the service is healthy, and from 18 to 24 min eu-west is really down. Alerting on one failing region after one failing tick pages at 6 min for nothing and again at 18 min for the real outage: FALSE PAGE, 1 false, 1/1 caught with a 0 min delay.

Quorum hides a regional outage

Raise Alert when failing regions reach to 2. The blip no longer pages, but eu-west is the only failing region, so the real outage never alerts: MISSED REGIONAL OUTAGE. At 3 the same happens. A quorum across regions is a test for global failure, not for a regional one.

Confirm from a second vantage point

Back at 1 region, tick Two vantage points per region (both must fail). The blip breaks only probe A, so us-east is not counted; the eu-west outage breaks both probes and alerts at 18 min: REAL OUTAGES CAUGHT. With 3 consecutive failing ticks the alert waits until 20 min, a 2 min delay.

Consecutive ticks and interval

Without the second vantage point, 5 consecutive ticks still page for the blip, which lasts exactly 5 ticks, and delay the real alert to 22 min. A 5 min Probe interval sees the blip once at 10 min and the outage at 20 min. In the Global outage scenario a quorum of 2 catches it; a quorum of 3 with 5 ticks every 5 min sees only 2 failing ticks and misses it. Probe-host blip with two vantage points stays QUIET.

Replay and limits

Any change recomputes the run and rewinds it; the Complete-run preview states the outcome at once. Play and Pause walk through the events, Next event jumps one event, Restart rewinds, Speed runs from 0.25x to 2x and Inspection time seeks from 0 to 60 seconds, where the outcome and final metrics appear. Incident windows are fixed and probes fail exactly inside them; no endpoint is checked. Alert routing, flap suppression and probe timeouts are not modeled.