Technical Note: LAKEer on the FACTS Grounding Benchmark

What is the FACTS Grounding Benchmark?

The FACTS Grounding Benchmark is an evaluation framework developed by Google DeepMind, Google Research, and Kaggle to measure the ability of AI systems to generate factually accurate, long-form responses strictly grounded in provided source documents. Introduced in December 2024 and hosted on Kaggle, it has become one of the most rigorous public tests for hallucination resistance in the AI industry.

The benchmark comprises 1,719 carefully constructed examples — each pairing a source document (up to 32,000 tokens), a user request, and a strict grounding requirement — spanning domains including finance, technology, medicine, law, and retail. It tests not just short-form factual recall, but the harder problem of synthesizing long-form responses that are simultaneously complete, relevant, and fully attributable to the provided document.

Evaluation Methodology

Every response is judged in two sequential stages:

• Eligibility — Does the response adequately and substantively address the user's request? Responses that deflect, hedge excessively, or fail to engage are disqualified outright.
• Factual Accuracy — Are all claims in the response fully grounded in the provided document, with no hallucinations or unsupported assertions?

Three frontier LLM judges — Gemini 1.5 Pro, GPT-4o, and Claude 3.5 Sonnet — score each response independently. Results are aggregated to reduce individual model bias (models are known to score their own family's outputs higher by a documented ~3.2%). The final score reflects the percentage of responses that pass both stages across the full dataset.

The dataset is split into 860 publicly available examples ("open set") and 859 privately held examples ("blind set"). Official leaderboard scores are averaged across both splits to prevent gaming or overfitting.

Current Leaderboard — Top 5 LLMs

The following rankings are from the FACTS Grounding Leaderboard on Kaggle (last updated November 16, 2025), with results independently reproduced by Kaggle:

A few things worth noting. The spread between #1 and #5 is only ~5 percentage points, reflecting how well-calibrated and difficult the benchmark is — there is no easy path to padding the score. All top performers are large, frontier-scale LLMs with massive parametric training and extensive RLHF optimization. The confidence intervals (±2–3%) are meaningful: differences of 2–3 points between adjacent ranks are statistically within noise, making the benchmark genuinely discriminating only at wider separations.

LAKEer's Results on the Public Dataset

LAKEer's 77.7% Accuracy on the public dataset places it above the current #1 LLM on the leaderboard (Gemini 2.5 Pro at 74.3% overall), a result that merits careful interpretation. The official leaderboard score averages public and private splits, while LAKEer was evaluated only on the public set — a direct apples-to-apples comparison would require running against the private set as well. That said, the margin is meaningful: LAKEer outperforms the top LLM on the same 860 examples by approximately 3.4 percentage points.

What makes this more significant is what LAKEer is. The FACTS Grounding Benchmark was designed to stress-test the raw generative grounding capability of individual LLMs — systems that produce responses directly from parametric weights, with no external verification layer. LAKEer is architecturally different: a neuro-symbolic enterprise knowledge platform that applies a defense-in-depth approach to factual accuracy, enforcing grounding at multiple independent layers before any response is surfaced. No single layer is relied upon — each stage catches what the prior stage may have missed.

At the core of this is LAKEer's multi-stage verification pipeline which systematically validates every claim in a candidate response against retrieved evidence before the answer is finalized. This pipeline spans retrieval (multi-channel vector search and knowledge graph traversal via Personalized PageRank), synthesis, and post-generation verification, ensuring that the grounding constraint is enforced end-to-end rather than left to the probabilistic tendencies of a single generative model.

What further distinguishes LAKEer is that its verification pipeline does not operate against generic document text alone — it operates against pluggable domain ontologies and domain taxonomies that encode the semantic rules, entity relationships, and business constraints of specific industries. Through LAKEer's cartridge-console architecture, domain knowledge is loaded as a structured ontology: in oil and gas, for example, the system understands the hierarchical relationships between formations, wells, and reservoirs; in finance, it understands regulatory classification precedence over product descriptions; in healthcare, it understands clinical terminology with the precision required for safe information retrieval. These ontologies are not static — they are pluggable, meaning LAKEer can be deployed across verticals without rebuilding the core platform, with each domain cartridge bringing its own taxonomy, entity types, and validation rules.

This domain-awareness elevates LAKEer's grounding from document-aware to industry-aware. A general-purpose LLM grounding to a document has no knowledge of whether a claim violates an industry-specific business rule, uses a term inconsistently with its ontological definition, or contradicts a regulatory constraint that sits outside the document itself. LAKEer's domain ontologies bring that layer of structured intelligence into the verification loop — catching errors that document-level grounding alone cannot detect, and ensuring that responses adhere not just to what the document says, but to what the industry knows to be correct.

Complementing this, LAKEer's forthcoming automated reasoning capability (currently in development) will further strengthen factual grounding by enabling structured, multi-step inference over the enterprise knowledge graph. This goes beyond retrieval-and-verify to reason-and-verify: LAKEer will be able to chain logical steps across knowledge units, surface only conclusions that are provably supported, and produce auditable reasoning traces alongside every answer. This positions LAKEer to extend its accuracy advantage further as reasoning-intensive enterprise queries — compliance checks, causal diagnostics, multi-document synthesis — become the norm.

The 92% Eligibility rate is equally telling. A 92% rate means LAKEer almost never failed to produce a substantive, on-target response. The eligibility filter disqualifies responses that hedge, deflect, or fail to engage with the user's request — behaviors that can artificially inflate accuracy scores by producing safe but uninformative answers. LAKEer's high eligibility demonstrates that its defense-in-depth verification does not come at the cost of response completeness, a tradeoff that simpler retrieval systems consistently fail to navigate.

What This Means

These results validate several core architectural principles behind LAKEer:

• Grounding is a system architecture problem, not just a model problem. The leaderboard demonstrates that even frontier-scale LLMs with extensive RLHF optimization top out in the low-to-mid 70s on factual grounding. LAKEer's defense-in-depth architecture — combining multi-channel retrieval, knowledge graph provenance, and a multi-stage verification pipeline — achieves higher accuracy by structural design rather than by model scale.
• The verification pipeline is the differentiator. Where LLMs rely on training-time alignment to suppress hallucination, LAKEer's HalluClean pipeline enforces grounding at inference time, independently of what the underlying LLM "wants" to generate. This makes accuracy a guaranteed property of the system, not a probabilistic outcome of a single model.
• Eligibility reflects enterprise fitness. For enterprise deployments, a non-answer is often as damaging as a wrong answer. A 92% eligibility rate means LAKEer consistently delivers substantive, request-aligned responses — a first-class requirement for production AI workflows where incomplete answers carry real operational cost.
• Pluggable domain ontologies make grounding industry-aware, not just document-aware. LAKEer's cartridge-console architecture loads domain-specific ontologies and taxonomies — encoding the entity relationships, business rules, and regulatory constraints of each vertical — directly into the verification loop. This means LAKEer catches errors that document-level grounding cannot: a term used inconsistently with its ontological definition, a claim that violates an industry business rule, a classification that contradicts regulatory precedence. It also means the platform is extensible across industries without rebuilding from scratch, a critical property for enterprise deployment at scale.
• Automated reasoning will extend the advantage. As LAKEer's structured reasoning capability comes online, it will bring multi-step, auditable inference over the enterprise knowledge graph to bear on complex queries — moving from retrieve-and-verify to reason-and-verify. This is the natural next frontier for hallucination-resistant enterprise AI, and a capability that generative LLMs alone cannot replicate.

DaaX is working to get its FACTS grounding benchmark results published on Kaggle; until then interested parties can contact DaaX for additional technical details on LAKEer's FACTS Grounding benchmark results.