Episode 30 — The FACTS Benchmark Explained: How Google DeepMind Tests Whether LLMs Tell the Truth

Episode 30 and a Recommendation: Signalful

Sunil: Okay, welcome to The Token Drop. I think this is Episode 30. A couple of things before we get started. Gokul Rajaram, whom I've known for a few years, is now a VC. He's a really well-connected guy and very helpful to startups. He texted me today to say he has a new site called Signalful, spelled just like it sounds: S-I-G-N-A-L-F-U-L dot com. It has interviews with founders, investors and operators, plus a newsletter. I looked at it, and it looks pretty cool. I encourage our audience to check it out.

A Broader, Educational Format: AI Benchmarks

Sunil: We got some feedback on our last podcast. People really liked that it was more educational. Sometimes we do a deep dive into a technical topic, but they also want a broader industry view. So maybe we can try that this week and see how it goes. One area I thought we could talk about is benchmarks.

Sam: Yep.

Sunil: We published our own FACTS benchmark results earlier this year. There's another benchmark for AI-based SQL generation called BIRD, and I'm sure there are many more. I think it would help to start with the FACTS benchmark and educate people about what it is, because it's gaining popularity and will become really common going forward. Sajjad, you know a lot about it, since you've been working on it really closely.

What Is FACTS?

Sunil: What is FACTS?

Sajjad: FACTS is one benchmarking effort in a long series of efforts that have been going on for quite some time. Large language models have a very large cognitive surface, so you want to be able to test and probe them in different domains and for different operational use cases, and really see how they perform.

A Short History of Language Benchmarks: SQuAD, GLUE, HELM

Sajjad: To give credit where it's due, for as long as I can remember there have been benchmarks for testing the efficacy of neural networks. In language, there was SQuAD, the Stanford Question Answering Dataset. Then there was a newer, improved benchmark called GLUE, and there were MNLI, SNLI and lots of others. When LLMs came around, Stanford aggregated many of these benchmarks into something called HELM, and HELM was the go-to benchmark for a long time.

Google DeepMind and Kaggle's FACTS Suite

Sajjad: Then Google DeepMind and Kaggle announced a benchmark, I think close to two years ago. It has been very thoughtfully put together to test the broad capabilities of LLMs and rank them against each other. FACTS is one of those benchmarks.

The Four FACTS Benchmarks

Sajjad: Under FACTS, there are about four different benchmarks. There's the FACTS Multimodal benchmark, for testing the multimodal capabilities of large language models.

Sajjad: Then there's the FACTS Parametric benchmark. How much knowledge do these models store, and if you quiz them, how well do they recall it? It's literally like a general knowledge quiz, a closed-book Q&A test.

Sajjad: Then there's FACTS Search. It's about a model's ability to use search tools: given a question, search the web, find the right answer, bring it back and provide it.

Sajjad: And the last one is called FACTS Grounding. Given a context and a question, can you accurately answer questions about that context? We've been using FACTS Grounding in our benchmarking efforts.

How FACTS Grounding Works

Sunil: That's interesting. Take the BIRD benchmark, which I know a little about. They give you a question along with the context you'd need to answer it, say domain knowledge or the company's language. Does the FACTS benchmark give you something similar?

Sajjad: Indeed. FACTS is a well-thought-through benchmark, and it covers many domains our customers operate in: legal, financial, technical and high-tech, and medical. I forget one or two other domains. In each domain, they've created fairly long passages, handcrafted by many experts. The passages are a lot like the ones on the SAT. The passage is given to the LLM along with a question, and you're required to answer that question.

Two Judges That Must Agree

Sajjad: Then there are two judges. In the past, there were three. Version 2 of the FACTS Grounding benchmark uses two judges, and both have to agree on the correctness of the answer and how comprehensive it is.

Sunil: So they give you a document, so to speak, and a question, and you're supposed to answer the question using that document and that document only?

Sajjad: Yes, indeed. As simple as it sounds, it's been a grueling benchmark for us, honestly. I don't know about the LLM providers. We chose to compete in it because we can also take documents as input and answer questions about them, much like a search engine.

What Makes It Grueling

Sunil: What's grueling about it? What's so tough?

Sajjad: First of all, there are about 920 questions, of which 860 are public. You can download the passages and questions, which is called the dataset. Then you give the passages to your system one at a time. In our case, that's LAKEer; in the case of an LLM, you provide one passage at a time to the model. Each passage has questions associated with it.

Sajjad: What's grueling is that two judges have to agree that the answer is eligible, which means it's comprehensive. On top of that, they judge the answer for factuality. Each passage contains a lot of facts, and in most cases they expect long-form answers. So you're aggregating and combining multiple facts from across the document and formulating an answer based on them. If even one of those facts isn't right, you can be penalized.

How the Judges Decide

Sunil: But how do the judges know? They're just LLMs. Do they use their inherent knowledge to say, "This looks nice," or do you give them the passage?

Sajjad: The judges are given the passage, the question and the answer, and they're required to judge that answer. They have to make sure all the facts relevant to the question are covered. If a question requires you to aggregate five facts from the document, all five have to be in the answer you generate.

Mandated Judges: Gemini Flash and GPT-5

Sunil: And I assume they specify which LLM judges you use. Otherwise you'd have an incentive to choose the easiest, loosey-goosey one.

Sajjad: No, no. The benchmark mandates two LLMs: one is Gemini Flash and the other is GPT-5. Those judges are pretty rigorous, and they do their job very well. They stick to their mandate. There are prompts for those LLMs, and they're really well-crafted, which makes the judges quite effective.

Public vs. Private Datasets

Sunil: We don't know what's in the private dataset, but we're assuming it's similar to the public one.

Sajjad: It should be somewhat similar, that's right. We looked at the questions and the scores, and for most competitors, the scores on the public dataset are comparable to the scores on the private dataset. If a model scores 82% on the public dataset, it also scores close to 82% on the private one. We don't have access to the private dataset, only the public one.

Domain Knowledge, or Reading Comprehension?

Sunil: We've talked about this in other podcasts: domain knowledge puts pressure on LLMs and causes them to hallucinate. These look like domain questions. In the FACTS benchmark, do they give that domain knowledge to the LLM?

Sajjad: Right now they don't. They don't expect you to have deep domain knowledge. These are analytical questions, just like the SAT. SAT passages could be about road safety, the Constitution or legal matters, but they're not so deep that you need specialized knowledge. Effectively, what they're testing is your ability to comprehend facts, reason with them, analyze the question and provide a reasonably comprehensive answer.

Long-Form Answers and Instruction Following

Sajjad: First, they expect a long-form answer. Second, they expect the answer to follow the instructions provided. Each question might have its own instructions. Some say, "Provide the answer as a bulleted list numbered 1 to 10." Others say, "Answer in no more than 2,000 words."

Sunil: Really? Wow, that sounds challenging. I would have failed that SAT test, man.

Sajjad: Or sometimes they'll say, "Use this many words and this many sentences, don't go beyond that, and make sure you cover all the facts." So there are all these instruction-following and formatting considerations.

Matching the Source's Tone and Intent

Sajjad: Then there's this. You know how it is when you sit down to write a long essay. These aren't quick, fly-by-the-seat-of-your-pants one-word or one-sentence answers. The moment you write a long answer, you expect it to be thoughtful, factually accurate and complete. It should also express the same sentiment, tone and tenor as the source, including superlatives. If the original document says a particular knife is one of the better knives, your answer can't say it's the best knife, because now you're violating the superlative. So you want the answer to be moderated, not swinging to the extremes, and to represent the tone, tenor and expressiveness of the original document, without using the exact same words but expressing similar sentiment.

Sunil: Intention.

Sajjad: Similar intent. Yes, intent is the right word. You have to communicate the exact same intent, and that's also judged.

FACTS Grounding v1 vs. v2

Sunil: I think you told me at one point they switched to a version 2. What changed between version 1 and version 2?

Sajjad: In version 1, there were three judges. In version 2, they reduced it to two. The dataset remains the same. I haven't evaluated the prompts themselves, but I think they may have changed a little. The dataset is the same, though.

The Leaderboard: No Model Near 90%

Sajjad: And it's still as challenging. If you look at the scores, even the best state-of-the-art models, like Sol, don't come anywhere close to 90%. Almost all of them are in the 80-82% range, and some models are at 59, 60 or 70%. They're all over the place, but nobody has gone much past the mid-80s. I think the leader right now is Sol, at about 85.2. I haven't seen anyone else cross that threshold.

The Brilliant Lawyer Who Makes Things Up

Sunil: Sam, do you want to say anything? It looks like you've been wanting to.

Sam: Sajjad covered most of the concepts. Some of our listeners aren't technical, so let me give a very high-level, non-technical overview of Google's FACTS benchmark. Say you hire the best lawyer in the world. Harvard, 40 years of wins, speaks like a textbook. There's one catch. Roughly one time in three, somewhere in that beautiful brief, a fact is simply made up: a date, a number. You won't know which one. Would you sign the retainer? That lawyer is the best AI model on the planet today, and we finally have an exam that proves it. Google calls it FACTS.

Does It Tell the Truth? Four Exams as a Day in the Life

Sam: Why is this exam different? Every other benchmark asks whether the model is smart. Can it do math, write code, pass the bar? FACTS asks the only question that matters once you put the thing into production: does it tell the truth? Not sometimes, not mostly. Every claim in every sentence in every answer. And it checks four completely different situations, because a model that's honest in one can be a liar in another.

Sam: Here are the four exams, told as a day in the life. Picture a new analyst on their first day. In the morning, the boss asks a question from across the room. No laptop, no notes; answer from memory. That's parametric. Mid-morning, they get a laptop and a search bar: now go find it and get it right. That's search. In the afternoon, someone drops a photo on their desk: what is this, and what do you know about it? That's multimodal. At the end of the day, here's a 40-page contract. Summarize it using only what's in there, without adding a single thing you think you know. That's grounding. Most models ace one or two of these and bomb the rest. The final grade is the average, so there's no hiding.

Numerical and Logical Reasoning Traps

Sajjad: Right, Sam. And for us, we intuitively know how to answer those questions.

Sam: Exactly.

Sajjad: But what you don't realize is that sometimes you're doing numerical reasoning or logical reasoning. You're assimilating facts and drawing conclusions from them. The questions are intelligently designed to see whether you'll reach the wrong conclusion or perform incorrect numerical or logical reasoning. They're designed to surface even minor infractions. It's frustrating the first time you run it. You keep wondering, "Why did my system fail this question?" Then you look carefully and realize, oh my god, and there are 860 of them.

Non-Determinism, and Why DaaX Verifies Every Claim

Sajjad: And the best part is that LLMs aren't deterministic. Every time you run the same question, you might get a slightly different variant of the answer. However different the answer is, does it still comply with the ground rules? That's extremely hard. I can't imagine how an LLM can pass all these questions. In our system, we make a lot of effort to ground answers, rewrite them and verify every claim. If a claim is incorrect, we go back and regenerate either that claim or the whole answer, then double-check and re-verify. That's why I feel sorry for the LLMs that have to take this test.

One Wrong Date, Zero Credit

Sam: You're absolutely right. One of the multimodal questions shows an orange locomotive and asks what it is and when it was introduced. The model's answer is gorgeous: right railroad, right model, right museum, right city in Mexico. But when was the train introduced? The model says 1939. It was actually introduced in 1949. Score of zero.

Sajjad: Yeah.

Sam: All that hard work, and you don't get half credit for being mostly right. That's the beauty of it, because that's exactly how it feels when AI burns you. Everything was perfect right up to the one detail you actually needed, and that one it invented. FACTS grades the way users feel. That's why I like the way it tests these things.

Eight Items on the Table, One in the Pocket

Sajjad: Exactly. Here's another example. Somewhere in a passage, it said something like, "In addition to the eight items on the table, I had another one in my pocket." It's plain English, but you have to read that passage as indicating there were nine items in total. You and I know that very well, but it's hard for an LLM, because the number 9 doesn't appear anywhere in the document.

Sajjad: With great difficulty, we've trained LLMs to do some arithmetic when they see purely symbolic equations, like 1 plus 2 equals 3. But numerical reasoning like "eight items on the table and one in my pocket, so nine in total" isn't intuitively easy for LLMs.

Sorting Kitchen Knives and the Cognitive Surface

Sajjad: It's fascinating, the kinds of questions they ask. Sometimes they ask you to sort products by price or capability. I remember one question asking to enumerate all the high-quality kitchen knives and then sort them by price or capability. For us human beings, that level of reasoning is intuitive.

Sam: Exactly.

Sajjad: For LLMs, it isn't easy. Some of the best LLMs fail at it. We realized we really have to separate language skills from numerical processing skills and from knowledge retrieval skills, then combine all those capabilities to formulate the answer. That's when I realized how vast the cognitive surface is, and how much you're exercising a whole collection of capabilities to answer questions like those.

Parametric Scoring: Accuracy, Hedging and F1

Sam: Exactly. And among the four cases you mentioned, grounding, parametric, search and multimodal, parametric has about 2,104 short questions, every answer backed by a Wikipedia page, so the fact was almost certainly in pre-training. They look at accuracy, at hedging, at attempted accuracy, and at the F1 score, the harmonic mean of the accuracy measures. It's a very well-put-together suite.

Where the Benchmark Falls Short: LLM Judges and Bias

Sam: But I still think the benchmark has some issues. The judges are models, and as Sunil brought up, many are Google's. Three of the four slices are judged by Gemini models; only the grounding slice adds GPT-5. And the paper reports judge agreement with humans at a macro F1 score of 65 to 78, which means roughly one verdict in four or five disagrees with a human. Self-preference bias is real. The two-judge setup on grounding reduces it, but doesn't remove it on the other slices.

Sunil: So there's room for improvement.

Grounding Rewards Refusals

Sam: Yes. And grounding rewards a refusal. If a model says, "The document does not state this," that counts as grounded and eligible. That's the right behavior for RAG, but it means grounding doesn't reward knowing more. Grounding is about stating only the facts in the document, which is why you pair it with parametric before concluding how good a model really is.

FACTS Grounding Is Effectively a RAG Test

Sunil: You mentioned RAG. So when these LLMs get the passages, it's a RAG implementation behind the scenes?

Sajjad: Yes, exactly. These are long passages, up to 32,000 tokens, so it's literally RAG. Effectively, the leaderboard you see for FACTS Grounding shows the results of a RAG test—

Sunil: Using that LLM.

Sajjad: Using that LLM, across those 860 questions.

Domain-Specific Benchmarks: Legal

Sunil: This is a general knowledge benchmark. In one of our meetings, Sajjad, or maybe Sam, you talked about domain-specific benchmarks people are developing, like a legal benchmark. Is that the same idea, except all the passages are legal?

Sajjad: I think the idea would be similar, with legal passages. There's a lot of material in the legal domain; there's something called case law. If you go to a lawyer's office, behind the desk you'll see a huge cupboard filled with hundreds of big volumes. The last time I met my lawyer, I asked her, "Did you read all of these?" She jokingly said, "Yes, every single one of them." There's no way anyone can read that volume of case content. They use them as references. There's a case, the questions, the arguments and everything around it. You could take any of those case law documents and frame questions around them. Under these circumstances, was the judge right? What would your argument have been? How would you counter it? I think that's how a legal benchmark might work. I haven't explored it, but that's something we could do next.

Legal Vocabularies, Ontologies and Harvey

Sajjad: We could arm our technology with a legal vocabulary and see how it performs. If it doesn't do well, we have another option: ingest a legal ontology and use it to guide the answering. We have several levers there. The leader in this space is a company called Harvey. I think they've taken one of the Chinese models, Kimi K2, and they're training it on case law and other legal documents. At the end of the day, it's still an LLM, and it may be exposed to the challenges LLMs have. It'll be interesting to see what benchmarks they've published and where they stand on them.

Small Models Winning on Truthfulness

Sam: I want to mention one important thing here: the plot twist. Small models seem to be winning, which nobody expected. The number one model on the leaderboard wasn't the biggest, most expensive reasoning model. It was Gemini 3.7 Flash, the fast, cheap one. It beat Gemini 3.1 Pro and GPT-5.5.

Sunil: You're talking about FACTS? Because on the FACTS leaderboard, the leader is Sol right now.

Sam: Right now, of course, yes. Previously it was like that. What I'm getting at is that the budget model was the most honest model. You don't necessarily need the latest and greatest, because being smart and being truthful are different muscles. That's something to understand about the FACTS benchmark: it measures whether you're truthful, as opposed to smart. The big reasoning models are trained to be confident, thorough and complete. That's wonderful when they know the answer, and exactly the wrong instinct when they don't.

Sam: So the fundamental question behind the FACTS score is whether the model is telling the truth. Of course there's reasoning and complexity involved, but the answer needs to be grounded. It has to come from the facts provided, not from what the model already knew. That's very important when you're talking about LLMs.

Sajjad: So, Sam, you're saying they might be too good for their own good.

Sam: Exactly. That's what I'm trying to say.

Wrap-Up

Sunil: I think we'll have to leave it at that, because we've run out of time. Thank you, that was a very good discussion.

Sajjad: Bye, guys. Thanks a lot.

Sunil: Bye-bye.