Flaky Test Detector

---
name: flaky-test-detector
description: Diagnose the root cause of an intermittently-failing test (timing assumptions, shared/leaked state, test ordering, real concurrency bugs) rather than recommending retries or skips. Use when a test fails non-deterministically.
---

# Flaky Test Detector

A flaky test is either a bad test or a real bug wearing a disguise. Assume
the latter until you've ruled it out -- flakiness in a test that exercises
concurrent code is often the only signal a real race condition will ever
give you.

## Common root causes, check in this order

1. **Timing assumptions**: fixed `sleep()` calls, assuming an async
   operation finished, assuming clock/timestamp values won't collide.
2. **Shared state leakage**: global/module-level state, a shared database/
   file the test doesn't clean up, tests that depend on run order.
3. **Unseeded randomness**: random data generation without a fixed seed,
   so some runs hit an edge case others don't.
4. **Real concurrency bugs**: the code under test has an actual race
   condition, and the test's flakiness is just exposing it non-deterministically.
5. **External dependency flakiness**: network calls, real clocks, or a test
   double that isn't actually deterministic.

## Process

1. Run the test repeatedly (or review its failure history) to characterize
   the failure -- does it fail more under load, in a specific order, or
   randomly regardless?
2. Read the test and the code under test together, looking for the
   patterns above.
3. Identify the specific root cause with evidence, not a guess.

## Output

The root cause, evidence for it, and the fix -- which is almost always
"remove the timing/ordering assumption" or "fix the underlying race," and
almost never "just add a longer sleep" or "mark it flaky and retry."