
Jul 16, 2026 / Updated Sep 30, 2026 RAIF: A Repairable Format for Structured LLM Output Today we're releasing RAIF, a format for structured input and output with LLMs. It's built for self-hosted agents and small models. On 10,677 real function-call payloads RAIF uses 9-10% fewer tokens than JSON (9.2% on cl100k, 10.2% on o200k). Tables and event logs save 25-37%, and wide repetitive structures up to about 70%. A single flat record comes out roughly even with JSON. Why Not JSON JSON output from an LLM is verbose, and to survive the model's mistakes you need a jsonrepair pass on top of it. RAIF does the repair in the format itself and reports every repair it makes. On cut-off output it recovers 46% of leaves at the same token budget, vs 41% for JSON plus jsonrepair. The decoder handles: self-repairing decode (markdown fences, slipped separators) truncation recovery (usable leaves from a cut-off stream) lossless, canonical round-trip to JSON raif-format: decode heals a broken stream live Running in vLLM RAIF now runs in vLLM, and your client code doesn't change. Install the plugin: `pip install raif-vllm` The plugin sits below your client apps. The model emits pure RAIF, and the plugin decodes it back to JSON at the server boundary. Your OpenAI client stays the same and never sees RAIF, but the model spends fewer tokens on the way. How many depends on the shape of the output, per the numbers above. One limitation: streaming `response_format` requests aren't decoded yet, so the client gets raw RAIF. Use non-streaming requests for structured output. Streaming tool calls work. raif-vllm: same OpenAI client, pure RAIF under the hood Three Independent Layers RAIF ships as three independent layers: raif-format (npm + PyPI libs) raif-lora (fine-tuned adapters) raif-vllm (the new plugin) Verified end-to-end on vLLM 0.19 across Llama-3.2-3B and Qwen3-4B/2.5-0.5B adapters. Try the LoRAs We have created 3 LoRAs to start with. You can play with them here: Llama-3.2-3B Qwen3-4B Qwen2.5-0.5B The 0.5B is the weakest of the three. It emits valid RAIF 97% of the time and is byte-exact on 81% of the held-out eval, short of the 95% bar the 3B clears. Qwen2.5-0.5B speaking pure RAIF Here are the repos with all the source code: github.com/skrrt-sh

Jul 16, 2026 / Updated Sep 30, 2026

RAIF: A Repairable Format for Structured LLM Output

Today we're releasing RAIF, a format for structured input and output with LLMs. It's built for self-hosted agents and small models.

On 10,677 real function-call payloads RAIF uses 9-10% fewer tokens than JSON (9.2% on cl100k, 10.2% on o200k). Tables and event logs save 25-37%, and wide repetitive structures up to about 70%. A single flat record comes out roughly even with JSON.

Why Not JSON

JSON output from an LLM is verbose, and to survive the model's mistakes you need a jsonrepair pass on top of it.

RAIF does the repair in the format itself and reports every repair it makes. On cut-off output it recovers 46% of leaves at the same token budget, vs 41% for JSON plus jsonrepair. The decoder handles:

self-repairing decode (markdown fences, slipped separators)
truncation recovery (usable leaves from a cut-off stream)
lossless, canonical round-trip to JSON

raif-format: decode heals a broken stream live

Running in vLLM

RAIF now runs in vLLM, and your client code doesn't change.

Install the plugin:

pip install raif-vllm

The plugin sits below your client apps. The model emits pure RAIF, and the plugin decodes it back to JSON at the server boundary. Your OpenAI client stays the same and never sees RAIF, but the model spends fewer tokens on the way. How many depends on the shape of the output, per the numbers above.

One limitation: streaming response_format requests aren't decoded yet, so the client gets raw RAIF. Use non-streaming requests for structured output. Streaming tool calls work.

raif-vllm: same OpenAI client, pure RAIF under the hood

Three Independent Layers

RAIF ships as three independent layers:

raif-format (npm + PyPI libs)
raif-lora (fine-tuned adapters)
raif-vllm (the new plugin)

Verified end-to-end on vLLM 0.19 across Llama-3.2-3B and Qwen3-4B/2.5-0.5B adapters.

Try the LoRAs

We have created 3 LoRAs to start with. You can play with them here:

The 0.5B is the weakest of the three. It emits valid RAIF 97% of the time and is byte-exact on 81% of the held-out eval, short of the 95% bar the 3B clears.

Qwen2.5-0.5B speaking pure RAIF

Here are the repos with all the source code: github.com/skrrt-sh