LettucePrevent - Real-Time Prevention of Factual Hallucinations in RAG

First of all, a huge thank you to my two supervisors Adam Kovacs and Gábor Recski!!

TL;DR:

• LettucePrevent integrates a token-level detector directly into the generation loop with a custom LogitsProcessor, outperforming state-of-the-art hallucination detection models (HDMs) under streaming inference at lower latency.
• Our regex-verifiable number detector achieves a >60% relative reduction in numeric hallucinations across all evaluated models with minimal latency overhead, ready to drop into any Hugging Face generate() call.
• LettucePrevent reaches a significant reduction (−26.6%) only on the model whose tokenization matches the Llama-3.1 boundaries it was trained on, at seven times the baseline runtime.
• Try it out yourself with Hugging Face's custom_generate module.

🤗 Custom Generate Module 🤗 LettucePrevent 68M HDM 💻 GitHub Repository 📜 Thesis Publication

The Mechanism: Modifying the Autoregressive Loop

Standard autoregressive decoding samples the next token directly from the model's logit distribution. LettucePrevent inserts one step in between:

1. Candidate Extraction: At each decoding step, the top-k candidate tokens are identified.
2. Hallucination Scoring: An HDM evaluates these candidates against the grounding context.
3. Logit Penalization: Candidates predicted to induce a hallucination receive a significant negative penalty in the logit distribution.
4. Modified Sampling: The model samples from the adjusted distribution, intrinsically suppressing unsupported content.

A skip_threshold parameter bypasses HDM evaluation when the top-candidate probability already exceeds the threshold, preserving baseline throughput on high-confidence steps. It is the dominant cost lever of the system, as the runtime results below show.

Why the detector has to be a decoder

Placing a detector inside this loop disqualifies most existing HDMs. Bidirectional encoders are trained to judge complete sequences, so every in-loop call confronts them with a streaming prefix mismatch. The 68M Ettin decoder (lebe1/lettuceprevent-ettin-decoder-68m-en) is causal instead. Since attention never reaches to the right, the prediction at position i equals the prediction the detector would emit if invoked on the prefix ending at i−1. Training therefore matches inference at no additional cost, because one forward pass over a full answer supervises every prefix simultaneously.

The Detector Toolkit

Under the streaming protocol on the 450-prompt benchmark, lettuceprevent reaches a binary F1 of 0.3886 on the hallucination class against 0.3556 for lettucedetect-base-modernbert-en-v1 and 0.2856 for tinylettuce-ettin-68m-en, at 11.6% and 37.3% lower cumulative runtime respectively. Note the absolute scale, since an F1 of 0.39 on a heavily imbalanced token grid remains a weak detector and everything downstream inherits that ceiling.

Integration via custom_generate

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mistral-7B-Instruct-v0.2"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", trust_remote_code=True)

context = "Revenue was 2400 million in 2021 and 3100 million in 2022."
question = "What was the revenue in 2022?"

text = tok.apply_chat_template(
    [{"role": "user", "content": f"{context}\n{question}"}],
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tok(text, return_tensors="pt").to(model.device)

out = model.generate(
    **inputs,
    custom_generate="lebe1/lettuceprevent-generate",
    trust_remote_code=True,
    tokenizer=tok,
    input_text=context,
    detector_type="number",       # or "lettuceprevent", "lettucedetect"
    skip_threshold=0.9,
    max_new_tokens=300,
)

print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Experimental Results

Evaluated on the RAGTruth dataset with the unique 942 summary prompts for the number detector on the train set and the 450 prompts for the factual evaluation on the test set. All factual hallucination counts are the verdict of lettucedetect-base-modernbert-en-v1 at τ = 0.7.

1. Number Detection

Logit-level interception filters unsupported numerical values before token selection on every architecture tested, without a single trained parameter. Latency stays around half a second per generation, between 5.3% and 9.1% relative overhead and amortizes further on larger backbones.

Two qualifications apply. The table reports the digit-form class the regex governs, and counting all numeric hallucinations including the word-form renderings some models fall back on, the reduction lands between 51.9% (Qwen) and 64.8% (Mistral). A residual of roughly two dozen digit hallucinations also survives per model, which the algorithmic contract should forbid, most plausibly because beam search at k = 4 lets a suppressed candidate re-enter through a competing beam.

2. Factual Detection (LettucePrevent)

Only Llama-2-7B produces a statistically significant reduction (Wilcoxon signed-rank, p ≈ 1.15 × 10⁻¹¹, r = −0.414). Qwen stays directional without significance (p = 0.069) and Mistral reverses the desired direction (p = 0.362).

The overhead column is a configuration artefact rather than a model property. The preceding sweep selected skip_threshold = 1.00 for Llama-2, invoking the HDM at every decoding step, while Qwen and Mistral landed on 0.99 and 0.80 and skip the majority of high-confidence steps. The winning configuration is therefore also the one that never skips, with a runtime standard deviation of 52 s and a worst case of 251 s per text.

Tokenizer alignment appears to be the binding constraint on the reduction itself. Lettuceprevent was fine-tuned against the Llama-3.1 tokenizer under the assumption that prevention transfers across tokenizers, and the results indicate it does not.

The neural path is therefore a research result that demands a tokenizer-matched host and an unattractive latency budget, while the number detector is production-ready as it stands.

Limitations

• Tokenizer alignment. Trained against the Llama-3.1 tokenizer. Hosts with divergent segmentation showed no measurable reduction, making tokenizer match a prerequisite.
• Dataset scope. RAGTruth annotates hallucinations post-hoc as spans over completed answers. This means tokens might only become unsupported in light of later content. To our knowledge no RAG corpus supplies prefix-conditioned labels.
• Computational overhead. A 68M forward pass at every decoding step is expensive, and skip_threshold is the only lever, yet the most successful configuration never skips.
• Recall vs. precision. At a 44.1% recall ceiling, false negatives permit hallucinations while false positives force the host into tokens it did not want.
• Derived numbers. The deterministic detector rejects any digit string absent from the context, so correctly computed values such as percentage changes are suppressed alongside fabricated ones. It suits extractive workloads, not arithmetic.
• Automated auditing. The auditor ranks Llama-2 as more hallucination-prone than Mistral, inverting the ordering RAGTruth was built to anchor, which may reflect a property of the auditor rather than the generators.
• Language. Optimized exclusively for English RAGTruth distributions.

Citation

@mastersthesis{Beccard:2026,
  title  = {Real-time Prevention of Factual Hallucinations in Retrieval-Augmented Generation},
  author = {Leon Beccard},
  school = {Technische Universität Wien},
  year   = {2026},
  url    = {https://repositum.tuwien.at/handle/20.500.12708/229242}
}