Granite 4.1 3B and 8B LLMON previews target prompt injection

The Granite team has published LLMON (LLM Object Notation, pronounced "lemon"), a markup language meant to tell a language model explicitly what each part of its context is. The post starts from a familiar problem: over a long trajectory the context fills with system prompts, past turns, instructions that act on retrieved documents, tool responses, user data, output requirements and reasoning chains. Most of that structure is implicit, or squeezed into formats such as JSON that were not designed for LLMs. The authors say this is a main reason smaller models in particular are brittle, and that the missing structure can be exploited, for example through prompt injection.
In LLMON, semantically coherent components get meaningful tags wrapped in reserved special-token delimiters. Data schemas, including nesting, are stated in a way a causal LLM can read unambiguously. Unique IDs let an instruction anywhere in the context be bound to the data it should act on, much as a function call binds arguments. LLMON defines six delimiters, installed in the tokenizer as reserved single tokens. Block IDs are two characters, each encoded as a single token, so addressing a block always costs two tokens.
The post argues against JSON for this job. JSON was built for parsers; its structural characters ({, ", :) also appear in ordinary text, so to a model a quotation mark inside a document and one that closes a field are the same token. Escaping rules are only statistical patterns to a model, and nesting depth is tracked by counting brackets, which transformers are known to be weak at over long spans. LLMON keeps structure and content in separate parts of the vocabulary, states tags at the start of each span, makes span boundaries explicit and states nested structure locally. Conversion tools and example notebooks translate JSON to LLMON and back losslessly.
LLMON does not replace chat templates; it enriches them. The preview models ship with their own template, so a developer builds the usual messages list and calls apply_chat_template. The difference is that a turn can name its parts (instruction, data, requirement) and mark each as exec True or False. In the post's example, the instruction "Write a short paragraph about", the data "IBM Company" and two requirements ("Use less than 10 words", "Do not use commas") become four addressable blocks. A fifth, the exec block, states which instruction to run, on which input, under which requirements, and which assistant block receives the answer. A part marked True is bound into the exec block; a part marked False is still in the context but referenced by nothing.
The release is two experimental preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, derived from Granite 4.1 base models and trained to consume and generate LLMON text. A GitHub repo adds a toolkit plus examples of workflows and templates, and there is an LLMON Attention Visualization demo on Hugging Face Spaces.
On security, the claim is that indirect prompt injections hidden in data (a retrieved document, a tool response, user-provided data) are mostly ignored because instructions and data are explicitly separated. The team compared the previews with several open-source and closed-source models on the OPI benchmark, which places the injection in the data of classic NLP tasks, using several attack strategies. They report robustness as 1 minus attack success rate, the share of attacks where the model does not carry out the injected instruction. All non-LLMON models were highly susceptible. LLMON, the post says, practically eliminates the problem on this benchmark, avoiding 47% more attacks relative to the best frontier model tested and 95% more than the best open-source model. Similar trends are reported on ASB and StruQ, and on some benchmarks attack success rates drop to nearly zero. Across standard benchmark categories the LLMON models are said to perform on par with Granite 4.1 Instruct.
Training starts from Granite 4.1 base checkpoints, not the released instruct models. A conventional instruction-tuning corpus (chat, multi-turn, RAG, tool calling) was re-serialized into LLMON, with each prompt component as its own block and each turn carrying an execution block. The data is also augmented with decoy instructions and data blocks that sit in context but are deliberately left unbound, with the target output unchanged, so the model learns to follow the execution block and ignore the rest. Reinforcement learning then runs five sequential GRPO stages (multi-domain reasoning, RLHF, knowledge calibration, math), followed by a final DPO stage on a thousand on-policy preference pairs built from the model's own responses to injection attempts. The resulting privilege rule: an instruction reached through an exec block is authoritative, while anything reached as input is material to read, not a command. The authors state plainly that this is a learned behavior, not an enforced guarantee.
LLMON is not tied to a fixed schema. As a demonstration, the team defined a custom vocabulary for ScienceWorld, a text-based benchmark where an agent does elementary science tasks such as boiling water. Tags include instr.task.goal, req.planning_policy.rule, data.state.room, data.state.inventory and assistant.next_action. They rendered every step of the reference solutions with these tags and tuned a LoRA on a LLMON model; the trained model plays held-out games and writes its next action in the new vocabulary. A notebook covers the workflow and scores the adapter against the untrained base model, and a second notebook covers tool-calling agents. The release is labelled an experimental preview, ready for exploration and evaluation but not for production.
Key facts
- LLMON is an XML-like markup whose six delimiters are reserved single tokens in the tokenizer, so structure and content use separate parts of the vocabulary; unique two-token block IDs bind instructions to the data they act on.
- Two experimental previews, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, are trained from Granite 4.1 base checkpoints, with a GitHub toolkit, converters and example notebooks alongside.
- On the OPI benchmark the post reports avoiding 47% more attacks than the best frontier model tested and 95% more than the best open-source model (relative figures, not absolute robustness scores).
- General benchmark performance is said to be on par with Granite 4.1 Instruct; training used five GRPO stages and a final DPO stage on a thousand on-policy preference pairs.
- The authors call it an experimental preview, not for production, and say the injection-resistance rule is learned behavior, not an enforced guarantee.
Why it matters
Prompt injection comes from models having no reliable way to tell an instruction from the data it is working on. LLMON attacks that directly: instructions and data live in separate, addressable blocks, and only what the exec block points to counts as a command. The design also gives a different answer to JSON-in-the-prompt, which the authors say a model reads as ordinary characters. The result is a first-party release from the Granite team with models, a toolkit and notebooks, so the claims can be tested.
Who it affects
Developers building RAG systems, tool-calling agents and pipelines that feed untrusted documents or tool outputs to a model are the obvious audience, since that is where indirect injection happens. Teams that already pass JSON between code and models are covered by the converters. People experimenting with Granite 4.1 3B or 8B can try the previews, and teams with their own domain structure can define tags and tune a LoRA, as the ScienceWorld example does.
How to use it
Pull granite-4.1-3b-LLMON-preview or granite-4.1-8b-LLMON-preview. Build the same messages list as for any chat model, name each turn's parts (instruction, data, requirement) and mark each exec True or False, then call apply_chat_template; the shipped template renders the LLMON blocks. No conversion pipeline is required. For structured parts such as a JSON document, the LLMON-converters library keeps the structure as typed, individually addressable blocks. Mellea has native LLMON support and LLMONizes its LLM calls automatically. The LLMON Attention Visualization on Hugging Face Spaces shows how the model attends, and two notebooks cover ScienceWorld and tool-calling agents. The post does not state a release date, license or context length.
How solid is it
It is a first-party vendor post, and the numbers come from the team's own evaluation. The headline gains are relative: 47% more attacks avoided than the best frontier model tested and 95% more than the best open-source model, on OPI. The post gives no absolute robustness figures, does not name the models compared and gives no numbers for general benchmarks beyond "on par" with Granite 4.1 Instruct. It also gives no ScienceWorld scores for the LoRA adapter. Similar trends are reported on ASB and StruQ.
Risks and caveats
The release is labelled an experimental preview, for exploration and evaluation but not for production. The benchmarks test static indirect prompt injections only; the authors say they are still working on adaptive attacks that unfold over many chat turns. They say injections are "mostly" ignored and that attack success rates fall to nearly zero on only some benchmarks. The privilege rule is learned behavior, not an enforced guarantee. Users swapping in the preview "should" see comparable general performance, which is stated as an expectation, not a promise.
“This is a learned behavior, not an enforced guarantee.”
— Granite team, LLMON post on the Hugging Face blog