ANALOGICAL PROMPTING · TRANSFER CHECK

Analogical prompting: test whether a pattern transfers

Borrow a design from a packet router or a batch import, map each element onto the real problem, and see whether the release tests notice what did not transfer.

Computed experiment · sixty inspection seconds

Your experiment

A ticket-routing design is borrowed from a packet router: route by a header field, fail over, hash each flow onto one path and drop packets when a queue is full. The release tests cover only routing and fallback, so the gate passes. Read which requirements are really met, then add the mapping check, combine analogies, test everything and change the target.

Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.

Event trace

    Computed data

    Final metrics

    The elements each analogy contributes and which requirement each covers or violates are a fixed fixture. The page models checking a transfer, not how well any model generates analogies; real analogical prompting asks the model to write its own exemplars, whose quality varies.

    Borrowing a design, element by element

    Analogical prompting asks a model to recall a similar solved problem and reuse its structure. This page checks the reuse. Source analogy supplies elements: the router routes by a header field, fails over, hashes each flow onto one path and drops packets when a queue is full; the batch import processes rows in order, checkpoints, upserts by key, retries failed rows at the end and pauses reading when its buffer is full. Target problem sets five requirements, Mapping check before use drops elements that map to nothing or violate a requirement, and Release tests decides what the gate runs.

    The opening run: a gate that tests the easy part

    For support-ticket routing, the router covers routing by product, fallback and per-customer order (flow hashing), but “drop when full” violates never dropping a ticket, and nothing covers safe redelivery after a crash: 3 of 5 requirements met. The release tests check only the first two requirements, so the gate passes a design that would lose tickets under load. Set Release tests to All five requirements and the gate correctly fails.

    Mapping check and a second analogy

    Tick Mapping check before use: the drop rule is discarded instead of imported, so nothing violates R4, but R4 and R5 are still not covered, 3 of 5. Set Source analogy to Batch CSV import alone: backpressure and checkpoint-plus-upsert cover R4 and R5, but routing is missing and “retry at the end” would reorder a customer’s tickets: 2 of 5. Choose Both with the mapping check: each analogy covers what the other lacks, the two harmful elements and the unmapped file-order rule are dropped, and all five requirements are met. With no analogy the direct attempt covers only R1.

    Failure boundary: closeness depends on the target

    Set Target problem to Nightly ledger reconciliation. Now the batch import is the close analogy: ordered processing, checkpoints, upserts and end-of-batch retries cover four requirements, and retrying at the end is fine here because the ledger only needs failures reported. The happy-path tests pass it while per-account routing is missing. The router alone meets 1 of 5, and its drop rule would break exactly-once application. Both, checked, under all five tests is the only setting that passes for the right reason.

    Reading the run and its limits

    Play replays the borrowed elements, the requirement check, the release tests and the outcome over sixty inspection seconds; the table shows which element covers or violates each requirement. Next event, Restart, Speed and Inspection time control the replay, and any input change recomputes it. The mapping is a fixed fixture that stands in for a reviewer’s judgment; it does not measure how good a model’s analogies are. The lesson is to test the requirements an analogy was not chosen for.

    Views are my own and do not represent any employer, past or present.

    Search