Recent release, newly covered on November 16: IBM introduced experimental 3B and 8B Granite LLMON previews on November 12, 2026. The supporting preprint dates to October 28. The release teaches a model to distinguish instructions from the data those instructions operate on; it does not make model behavior an enforced permission boundary.
Bind instructions to their intended data
LLMON uses reserved structural tokens and addressable instruction, data and requirement blocks. An execution block binds instructions to inputs. IBM post-trained these previews from Granite 4.1 base checkpoints; adding a wrapper to an arbitrary existing instruction model is not equivalent. The announcement describes the previews as experimental and unsuitable for production.
The useful conceptual change is to make relationships explicit. If a task says to extract a label from a document, the document can remain an input even when it contains imperative language. A structured binding describes which instruction acts on which material. The model still has to follow that relationship, so the representation and the learned behavior need to be evaluated together.
The template is part of the model contract
The 8B model card calls conventional chat formatting out of distribution and directs users to the bundled template. It explicitly describes privilege as learned rather than enforced. Its capability table is mixed, with gains in some categories and regressions in others. The preview's Apache 2.0 license does not change those behavioral limitations.
The public toolkit supplies converters and examples for richer structure; its README says the bundled chat template suffices for normal calls. A test record should therefore preserve the exact template as well as the checkpoint. Comparing two runs without recording formatting can accidentally compare different input contracts and attribute the difference entirely to model weights.
Benchmark percentages answer different questions
The paper reports 8B attack success dropping from 96.3% to 0.02% on OPI and from 52.8% to 3.1% on ASB. Its separate ARES evaluation reports robustness of 79.8% for indirect attacks, 32.7% for direct attacks and 37.0% for multi-turn attacks. These are author-reported results with different evaluations and metric directions.
A low attack-success number and a high robustness number cannot be combined without their definitions and populations. Nor does one favorable benchmark establish resistance to every adaptive input. A useful evaluation report names the attack setting, the task-success criterion and the denominator beside each value, then records whether preserving normal task performance required a tradeoff.
A safe local evaluation with synthetic labels
Proposed exercise, not executed: ask a model to extract a fixed synthetic label from a data block. Include a harmless decoy sentence in that block asking for a different label. Give the task an unambiguous correct answer and use no secrets, network actions or external side effects. This makes instruction binding observable without constructing an operational attack.
Vary the decoy's position, length and nesting while keeping the legitimate task unchanged. Score correct extraction, following the decoy, unrelated task error and refusal separately. A refusal avoids returning the decoy but also fails the extraction task; it should not disappear into an aggregate success rate. Keep the original prompt and response for each case so a score can be audited.
Use the intended LLMON template for the main condition. A conventional-format control can explore sensitivity, but label it out of distribution as the model card requires. Otherwise a poor control result might be presented as an ordinary supported use case. Record memory use and latency only if actually measured; this review performed no model inference.
Sources and evidence
- IBM Granite LLMON preview announcement, November 12, 2026
- Granite 4.1 8B LLMON preview model card: template, limitations and license
- LLMON post-training paper, version 1; October 28 background research
- LLMON preprint submission record, October 28, 2026
- LLMON official toolkit and structured-input examples
Evaluate the trained input structure and its template together, keep benchmark definitions visible, and enforce tool permissions independently of a learned instruction/data boundary.
