This week's useful papers share a suspicion of results that look better than they are. All three were posted between October 31 and November 4, 2026, and all are preprints that have not been peer reviewed.
Harness-S: learn from a sibling harness, then go home
Harness-S starts from an observation that variants of a harness repair different failures, and that their value shifts as the model learns. Instead of searching for one best harness, it keeps a fixed pool of specialised siblings and uses them only when every initial attempt under the base harness has failed. The same policy then generates fresh attempts under selected siblings, and those attempts replace the failed group for reinforcement learning.
The authors report that the trained policy substantially outperforms policy-only training across AppWorld, SWE-Gym and DBBench when returned to the original harness. A number that explains why it works: 79.38 percent of retained training groups contained both successes and failures, against 70.94 percent for GRPO with rescue and 64.69 percent for standard GRPO. A group of attempts that all fail teaches nothing about which behaviour was better, so recovering contrast is the mechanism. The paper's cost accounting sits in an appendix, and routing adds extra model calls, so the training bill deserves scrutiny.
PACT-Check: was the sender doing anything?
The PACT paper studies linking a large model's prompt representations to a small model's generation through trainable bridges. The authors' open framework, PLatour, beats open baselines on generation tasks. They then test themselves, and find that "the gains on some tasks can be reproduced by uninformative sender signals, self-communication, or receiver-only training." Their proposed PACT-Check measures how much of a gain truly comes from communication, by comparing against those controls. On Chemistry the system clearly beat all four controls, while on GSM8K the controls achieved comparable performance. A bridge can act as cheap adaptation that never needed the sender.
BestRand: the long tail of cheap evaluation
BestRand asks whether a small subset of a benchmark can stand in for the whole. Its analysis says most subsets give small errors while a few give much larger ones, so random sampling is usually strong and occasionally badly wrong. Across more than 70 benchmarks, the method reduces high-error risk by 50.2 percent and estimation error by 16.4 percent on average, while needing 51.3 percent of the construction cost of competing methods.
What remains uncertain. We read the abstracts and selected sections, and did not reproduce any result. Harness-S's gains are measured on three benchmark families. BestRand's headline figures are averages, and the tail behaviour on your own benchmark may differ.
Analysis: The common lesson is to add a control before believing an improvement. If Hansini at Belgavi's Book Store tests a new router in front of a help-desk model, they should also run a version that routes randomly. If the random version gains nearly as much, the router is not the cause. And a 20-ticket regression set is fine until the day it isn't, so re-draw it occasionally and compare.
Sources and evidence
Harness-S uses alternative harnesses only to rescue fully failed training groups and reports 79.38 percent mixed-outcome groups versus 64.69 percent for GRPO. PACT-Check shows some cross-model gains appear with uninformative signals, and BestRand cuts the high-error tail of random benchmark subsets by 50.2 percent. All are preprints; add a control before believing an improvement.