FEW-SHOT LIBRARY · EXAMPLE SELECTION
Few-shot library: select examples that cover the query
Choose prompt examples from a ten-example library by recency, shuffle or similarity, and see when the chosen set covers the ticket.
Event trace
Computed data
Final metrics
Ten fictional support examples with hand-assigned tags and token sizes. The "model" is a deterministic stand-in: it copies the label of the selected example with the most shared tags, or the most frequent label when nothing overlaps. Real models also weigh wording, order and instructions; this isolates the selection effect.
Recency is not relevance
The library holds ten labelled support examples, each with tags and a token size. The opening Incoming ticket, Half the order missing, wants money, carries the tags partial, missing and money; its gold label is refund. Selection strategy is Most recently added, Examples in the prompt is 3 and Example token budget is 300. The prompt receives E10, E9 and E8, 84 tokens, none sharing a tag with the ticket. The stand-in model falls back to the earliest of the tied labels and answers replacement: WRONG LABEL.
Select by similarity
Choose Highest tag overlap. E6, which shares all 3 tags, leads, followed by E1 and E4: CORRECT, 100 tokens. Best per label, then overlap takes the best refund, replacement and escalate examples, E6, E4 and E5, and is also CORRECT at 100 tokens. Back on recency, raising Examples in the prompt to 6 reaches E6 and becomes CORRECT at 186 tokens; Fixed shuffled order is still wrong with 6 examples because E6 is last. More examples only help if the right one gets in.
Budget and coverage gate
Under Highest tag overlap, lower Example token budget to 60: only E6, 38 tokens, fits and the answer is still correct. Under recency the same budget admits E10 and E9, 54 tokens, and stays wrong. Turn on Abstain unless an example shares at least two tags: the recency run now ends ABSTAINED and asks a human instead of guessing. The gate has a cost. Wrong colour received carries a single tag, so no example can share two and the gate abstains under every strategy, even when replacement would be right.
When similarity ties
Pick Double charged, wants a supervisor, gold label escalate. E3, a refund example, and E5, an escalate example, each share two tags. Highest tag overlap takes the earlier one, E3, and answers refund: WRONG LABEL, and the coverage gate passes because two tags are shared. Recency happens to include E8, which shares billing, and answers escalate correctly. Lamp arrived broken, wants another is correct under every strategy at 3 examples, under recency only because the fallback label happens to be replacement. Coverage of tags is not coverage of intent.
Replay and limits
Any change recomputes and rewinds. Play and Pause, Next event, Restart, Speed from 0.25x to 2x and Inspection time from 0 to 60 seconds walk through each addition or budget skip; the board shows which ticket tags are covered. The verdict appears at 60 seconds, and the Complete-run preview shows it at once. Tags and sizes are hand-assigned, and the model copies the label of the best-overlapping example. Real models also weigh wording, order and instructions.