On November 2, 2026, Anthropic's Frontier Red Team, working with researchers from EPFL, MIT and StarkWare, published "Progress on provable inference for large language models" together with a technical report, a protocol specification, code and Lean proofs. The system is called FlashProve. The problem it addresses is stated plainly in the post: a user "cannot check that the responses came from the specific model that a developer claims is responding, as opposed to one secretly substituted in by a routing service for cost reasons, or even by an attacker who has tampered with the token stream."
How the proof works
FlashProve has two phases. Offline, the model provider publishes a commitment to the weights, which the post describes as "a short fingerprint to pin down one exact choice of weights while the weights themselves remain hidden." Online, for each request, the provider produces a proof file for the statement that "The committed model produces these output tokens when evaluated." The proof is a zkSNARK: anyone holding the commitment, the prompt and the reply can check it without the weights, and the post says forging a valid proof of a false statement is "tantamount to breaking widely believed cryptographic assumptions". The proof hides the weights and which experts ran at each layer. It does not hide the architecture.
Anthropic says Claude produced the improvements to the proof system and that the team counters the difficulty of evaluating those decisions with "top-level auditable statements in Lean". Soundness depends only on the verifier, so the release includes a fast C++ verifier with no formal guarantee and a slower Lean verifier that is proved to accept only what the soundness theorem accepts. The authors report that the two agreed on every proof in their tests.
The measured cost
The test model is Llama 4 Maverick, a 400-billion-parameter mixture of experts with about 17 billion parameters active per token, in a public 4-bit conversion. The prover runs on one H100 with 24 CPU cores and about 290 GB of host memory, streaming weights in one segment of layers at a time. The offline commitment takes about 39 minutes and is good for about 16 million requests. A batch of 24 requests of 1,024 tokens takes about 223 seconds to prove, which is 110 tokens per second, and each proof is 18.78 MB. The verifier downloads a commitment of about 6.5 MB and checks a proof in 17 seconds on 24 threads, or 21 seconds on one.
The comparison with ordinary serving is the number to remember. Against vLLM running the FP8 release on eight H100s, proving a batch took "about 72 times longer than vLLM takes to process the same 24 requests at once (66 to 77 across runs)" per GPU, and about 94 times longer at the highest load tested. FlashProve proves after the fact rather than in line with generation, so a client gets its guarantee later than its answer.
What remains uncertain. The limitations section is specific. The proof covers "an exact integer version of the published weights" under greedy decoding, so a reply from a floating-point server, or from sampled decoding, "could depart at some position and no longer exactly match the computation proved". Contexts are limited to 1,024 tokens. The fast code is not formally tied to the Lean protocol, and side channels such as proving time are out of scope. The system relies on the model being a mixture of experts. We have not run the code, and the figures are the authors' own measurements.
Analysis: The near-term use is spot-checking rather than proving every reply. At roughly 72 times the serving cost, a provider could prove a small random sample of traffic, and a customer or evaluator could ask for proofs of chosen requests. At Belgavi's Book Store, Geetha's team buys help-desk model access through a router. If a provider ever offered this kind of proof, the useful contract term would be the right to request proofs for, say, one CASE conversation in a thousand, chosen by the buyer after the fact. The greedy-decoding limit matters here too: a buyer would need to know whether the served configuration is the one the proof covers, otherwise a failed check proves nothing about substitution.
Sources and evidence
Anthropic's FlashProve (November 2, Apache 2.0) generates zero-knowledge proofs that a committed set of hidden weights produced a reply. On Llama 4 Maverick it proves about 110 tokens per second on one H100, with 18.78 MB proofs that a CPU verifies in about 17 seconds, at roughly 72 times the per-GPU time of serving with vLLM. The guarantee covers an integer model under greedy decoding with contexts of up to 1,024 tokens, and the fast verifier is not formally verified.