USAGE METERING · DEDUPE AND VALIDATION
Usage metering: detect duplicates and invalid records
Run ten October meter records for one tenant through dedupe, validation, unit conversion and period assignment, and compare the invoice with the correct 6,200 tokens.
Your experiment
Ten records arrive: a client retry, a negative count, a synthetic probe, a legacy record in thousands of tokens, a call at 23:59 on 31 October received after midnight, a November call and two different calls of the same size. Start with no dedupe and no validation and add one control at a time.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, Speed changes the pace and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Fictional tenant, records and price ($2 per 1,000 tokens). The correct total is fixed by the fixture: each distinct real call in October by event time, in tokens. Validation rejects negative quantities and records marked test=true; Reject invalid records also rejects the unknown unit k-tokens, while the convert option multiplies it by 1,000. No billing system is called.
Errors that cancel out
Ten meter records arrive for tenant acme in October. The correct bill is 6,200 tokens, $12.40. With no dedupe and no validation the meter bills 5,602 tokens, $11.20. That looks plausible, yet it hides four separate errors: the retried call U2 is billed twice (+1,200), the probe U5 is billed (+500), the buggy negative count U3 subtracts 300, and the legacy record U6 in thousands of tokens counts as 2 tokens instead of 2,000. Totals that are close to expectations are not evidence that the meter is right.
Deduplicate on the event ID
Set Duplicate detection to By event ID: U2 carries the same event ID e1 as U1 and is dropped, so the total falls to 4,402 tokens. Choose By tenant and quantity instead and U2 is still dropped, but so is U10, a different call that happens to be the same size as U9: with full validation that key bills 5,300 tokens, UNDERBILLED by 900. A duplicate is a repeated call, identified by the ID the client attached, not a repeated number.
Validation and units
Under Record validation, Reject invalid records removes the negative count and the synthetic probe, but it also rejects U6 because k-tokens is an unknown unit: 4,200 tokens with event-ID dedupe. Reject invalid records and convert units converts U6 to 2,000 tokens and the invoice becomes EXACT at 6,200 tokens and $12.40. Validation that silently discards valid usage is as costly as no validation; unknown units should be converted or quarantined for review.
Which period a late record belongs to
U7 happened at 23:59 on 31 October and was received at 00:02 on 1 November. With event-ID dedupe and full validation, setting Assign records to a period by to Receipt time moves U7 out of October and bills 5,800 tokens; Event time keeps it and bills exactly 6,200. Record the event time at the source and close a period only after late records have had time to arrive, or you shift revenue between periods.
Reading the run and its limits
Play, Next event, Restart, Speed and Inspection time step through every record decision and the invoice across sixty seconds; the table shows each record, its note, the decision and the running total. The fixture is fictional and small, the price is flat and one tenant is billed. Real metering pipelines add late-arrival windows, corrections after a period closes, aggregation before rating and per-model rates.