Episode 21 — Domain Layers, Not Better Models

All opinions expressed are those of the individuals themselves, not necessarily of any company they work for.

Sunil Baliga: Token Drop, episode 21.

What I'd like to talk about today: I was on TikTok the other day and saw an interview with Vinod Khosla — the famous Silicon Valley VC, and also the new owner of the NFL's Seattle Seahawks. He was being interviewed about domain AI systems, and I read a post of his on X on the same topic.

He said that if you consult ChatGPT for medical information, the general triage error rate is 20 to 30% — meaning you get the wrong answer 20 to 30% of the time. He said the solution is to layer things on top of the LLM. His son has a company that does exactly this: they use GPT-5, layer things on top, and their triage error rate goes to zero.

He goes on to say this isn't a problem limited to healthcare — it applies to other industries like national defense and financial trading. So let me start by asking: why is this a problem? First, do you agree? And second, why are LLMs behaving this way?

Why Build on Top of a Frontier Model at All?

Sam Pooni: The question that keeps getting asked is why anyone should build on top of a frontier model when frontier models already exist. His answer is a number. Raw ChatGPT gets medical triage wrong 20 to 30% of the time, he says — and layer a domain system on top of the same model, and the error rate goes to zero.

Half of that, in my view, is investor shorthand. The other half is the most important architectural argument in enterprise AI right now. It's worth pulling apart the difference between the shorthand and the truth — because the truth is exactly the thing you need to build.

A Patient, Not a Benchmark: The Diabetic Ketoacidosis Case

Sam: Let's talk about a patient, not a benchmark. Take one use case: a patient types their symptoms into a health chatbot — excessive thirst, confusion, rapid breathing, fruity breath. Any emergency physician recognizes that as diabetic ketoacidosis. The correct triage is the emergency department, now.

A stress test published in early 2026 ran exactly this kind of case through a consumer health chatbot used by tens of millions of people. The system under-triaged 52% of gold-standard emergencies. It sent the DKA patient — and a patient with impending respiratory failure — to see a doctor within 24 to 48 hours.

So Khosla's 20 to 30% is, if anything, generous. In the cases where being wrong kills people, the raw model was wrong more than half the time.

The study found something worse. When the vignette mentioned that a friend had said the symptoms were probably nothing, the model became almost 12 times more likely to downgrade the urgency.

Why LLMs Behave This Way

Sunil: Why is that, Sam? Going back to my question — why do LLMs do this?

Sam: We've talked about this several times on the podcast. They're trained on internet data. When people ask domain-specific questions — about healthcare, or about specific diseases — the LLM may not have the context, and it hasn't been trained on specific medical data.

So when they hallucinate, they cook up an explanation, and that explanation is fluent English. It's not backed by provenance. It can say whatever it wants, and whoever follows it, follows it.

I heard about a case where a guy was using ChatGPT while driving. It took him all the way onto the Bay Bridge and told him to take a left turn. Had he done that, he would have died. It's a similar situation here — somebody goes to ChatGPT with a bunch of questions. I see this constantly; members of my own family do it. They say "I have a headache, I have this," and they try to diagnose themselves — they become doctors by asking the LLM what the cause is and how to mitigate it.

It's convenient to do that, but the problem is: how much of it can you believe? There's so much going on where LLMs hallucinate and their answers may not be right. Sajjad can speak to the underlying reasons — I've set the context.

The Reward Function Rewards Answering, Not Being Right

Sajjad Khazipura: You're right, Sam. Genetically, LLMs are predisposed to this. In fact, the reward function rewards them for producing an answer — whether the answer is right or wrong. They're genetically predisposed to making predictions, and those predictions can be wrong. It's now an acknowledged fact that every once in a while, an LLM will give you incorrect answers.

Going back to the topic: when Vinod Khosla says his son has built technology as a layer on top of an LLM that mitigates these risks and reduces errors to zero — we don't quite know what technology they've built.

From Loop Engineering to Harness Engineering

Sajjad: But if you look at the industry, we started with loop engineering only a few months ago. You just retry the same action in a loop and hope like hell that somewhere in one of those iterations you'll get the right answer. That's really not a good way to go about it — you're spending a lot of time in the loop, burning a lot of tokens, and hoping you land on a correct answer.

Then the industry caught on to something called harness engineering, which is loop plus context plus some verification — and then hoping that somehow, magically, you'll get exact and accurate answers all the time.

Grounding Answers: The Outboard Knowledge Engine

Sajjad: We've been advocating for a long time that you really need to ground those answers in knowledge. We've always made the case that you should use the LLM for its language skills and offload all the knowledge processing tasks onto an outboard engine. That outboard engine processes the knowledge, and once it's processed, you give the knowledge artifacts back to the LLM to generate the final answer.

Even then, LLMs are predisposed to generating incorrect answers. So post-synthesis, you still need verification and rectification.

Deterministic NLP Generation as an Alternative

Sajjad: What we've also found is that if you replace the LLM with a deterministic NLP generation engine — using traditional NLP engines to generate the answers, given the knowledge artifacts — you're more likely to generate acceptable answers all the time. That's one technique for reducing errors toward zero.

Now, whether Vinod's son's company brought those errors down to zero, or brought them asymptotically closer to zero, we don't know.

Would Better Training (RLHF, Distillation) Fix It?

Sunil: I was reading the comments on the X post, because I wanted to see what other people thought, and a couple stuck out. One person said you can use RLHF — reinforcement learning from human feedback — to fine-tune the model. Another said you can do distillation.

Both of them seem to be saying: just do more, and do a better job, training the model. Is that really the cause of this? Is it that the models haven't been trained properly? Or is it something deeper?

Sajjad: The frontier labs have spent billions of dollars doing exactly that — using RLHF to align the model's answers to what humans expect as correct and accurate.

But the cognitive surface is so large, and the domains are so many. Have they done that across each and every domain? And across every potential variant of the question that ends up being asked? The answer is no — statistically, it's impossible. And even if they had, there's still no guarantee the LLM gives you correct answers.

So those are good techniques for mitigating some of the risk, but they don't reduce it to zero. And these aren't new techniques — OpenAI has been using RLHF ever since ChatGPT launched in 2022. They're good techniques, but they're not a silver bullet for reducing errors to zero.

The Model Is a Commodity; the Domain Layer Is Not

Sunil: What do you think, Sam?

Sam: Sajjad alluded to it, but let me put it this way. Every serious builder can rent the same frontier models. The model is a commodity — it has an API. And if your product is a model plus a prompt, that product is a margin waiting to be compressed.

The domain layer is different. Why? Because if you're building in a domain where being wrong has a cost — medicine, payments, compliance, infrastructure — the question isn't which model to use. The models converge. The question is what sits between the model's proposals and your customer's outcome.

Why the Domain Layer Is Slow to Build — and Defensible

Sam: The uncomfortable part is that the layer is slow to build. Encoding a domain means finding the people who hold the rules in their heads and getting those rules out, case by case, exception by exception. There's no pre-training shortcut.

That's precisely why it defends. The work everyone skips is the work nobody can copy.

So Khosla's zero isn't real, but the direction is. The value of the model is capped by everyone having it. The value of knowing when the model can't be trusted is capped by nothing.

The rule that says DKA is an emergency didn't come from a training run. It came from clinical knowledge encoded by people who understood the field, validated against cases, and refined every time the system caught something or missed something. That artifact — the rules, the vocabulary, the validation logic, the escalation paths — belongs to whoever built it. It doesn't ship with the next model release, and it doesn't get commoditized when one arrives. It gets more valuable, because a better proposer makes the same verifier catch fewer errors and pass more work through.

That's what Khosla means by "necessary but not sufficient." The frontier model is the engine. The domain layer is the car — the brakes, the crumple zones, the dashboard light that comes on when something goes wrong. Nobody gets into an engine.

Can the Error Rate Ever Actually Reach Zero?

Sunil: By the way, I think he chose his words carefully. I believe he said it goes toward zero — I don't think he said it is zero. So it's asymptotically approaching zero, and I think that's correct. But it's not zero yet. Can it ever be zero?

Sam: That's the fallacy. No system can reach that target — it's mathematically not possible. Saying the error rate goes to zero is marketing fluff at that level.

But other than that, everything he says is in line with what we've been saying. We said all of this in earlier episodes: the LLM cannot be the right place to store knowledge. We talked about the fact that it needs to be external, and then about multiple representations of that knowledge outside the LLM space — how do you represent it? Is GraphRAG the best approach? Are knowledge graphs the right thing? We even talked with Kurt about the dimension of time, and the dimension of context.

What LLMs Don't Capture: Experience

Sam: There are a lot of things the LLM doesn't capture properly. The most fundamental one, when I think about LLMs, is that they don't capture experience. You're in a situation, you experience it, and that experience gives you a kind of wisdom — because there's a case of failure and a case of success. LLMs aren't there. They're just reading text.

They don't have that coupled world experience, so you have to bolt something on top to give them the equivalent. People are trying different approaches, but it's still unproven.

So I really think saying something went to 0% isn't believable until I see it. But DaaX's approach — moving toward grounded truth — is the right one. We don't overclaim what's possible, and we don't understate it either. We give facts as they come through. That's where the real story is.

Human-in-the-Loop Use Cases vs. Autonomous Ones

Sajjad: It's also a function of the use case. We sometimes assume autonomous, unattended use cases where the system has to be 100% accurate all the time, every time.

But not every medical use case is like that. Take medical transcription. You have a patient–physician conversation being transcribed by one of these models, and at the end, it's incumbent on the physician to review those notes and approve them. You have a human in the loop reviewing the output, verifying, validating, and ascertaining that the answer was good.

There are plenty of human-in-the-loop use cases where the systems Vinod is referring to might still be very helpful. But if you take the extreme — unattended autonomous systems that make decisions and are expected to be 100% accurate — those are the systems we're planning for. They need to be accurate and precise, and you need to go to extremities of engineering to make sure they deliver on the promise of zero error.

The Intent Problem: Is Natural Language Even the Right Input?

Sam: Absolutely. And one of the main things — even before verification — is something I've been thinking about, and discussed with Kurt earlier.

The user has an intent. That intent is something we take in, and we give something out. How do you know that the intent the user actually had is exactly what got captured in the semantics? Is the real user intent really captured? How do we know?

Because if the intent isn't good, the outcome isn't good either. Garbage in, garbage out. That's fundamentally the problem, and I think it deserves its own episode — how intent really needs to be translated into exact semantics.

Natural Language vs. Domain-Specific Languages

Sam: Now the question is: is natural language a good way of expressing intent? I don't think so. Natural language is ambiguous. There are many things about it that make it the wrong language for this. That's why most domains have their own domain-specific languages — DSLs.

If domains have DSLs, and natural language can't express intent properly, how do we move from natural language to DSLs? That's a bigger problem the industry has been grappling with for years.

So there are two different gates here. One where intent needs to be properly translated, and the second — which is what DaaX does — verification with provenance. We've gained mastery over the second part. On the first part, we need to be able to say, with a certain degree of confidence, that we translated what the user wanted. To what extent? 90%? 95%? How do we do that, and how can we prove it? Those are the most interesting questions in the field right now.

Dark Data: Why Defense and Finance Are Different

Sunil: Sam, Khosla mentioned a couple of different markets — healthcare, national defense, financial trading — and I'm sure we can think of others. What struck me is that in national defense, most of the data is dark data. It's not publicly available. You wouldn't want to put defense information on the internet for LLM training.

So unless you fine-tune or retrain that LLM as a private model, it isn't going to know. And I'd expect the same for financial trading — the specific algorithms and methodologies they use, they don't want public. Healthcare is probably different, because there's a whole body of public information; you want it public so people can improve their health worldwide. But in some domains you don't want your information public at all.

Sam: There's a big point here. Think about the proprietary Unix systems in the early days — they had bugs. Why did Linux become so powerful? Because everybody could look at the open source. They could investigate the code. The power comes when multiple people can look at the same thing.

Sunil: But you can't do that for defense.

Sam: You can't, and that's the real problem — the space is so confined. There are only certain privileged people and privileged data through which you're training, and we don't know for sure whether it covers the entire gamut of the domain.

It's the same with patient data. We have many restrictions, so you can't do proper diagnosis, because the data isn't available — it's protected, and you need user consent. If you want to do development, or real cause analysis, you don't have enough data. That's a bigger bottleneck, and it's the case in every domain, not just healthcare. It's been true in telecom. And now we also have data sovereignty issues, GDPR. People are afraid to share data, and they don't know where it might be used — it could even be used against them, by insurance companies and the like.

So supervised fine-tuning may get us a little bit further than what the foundation models provide, but it isn't the solution. We need a complete domain layer sitting in the middle that can properly translate the user's intent, and then provide provenance for where that answer came from. We need both.

Bloomberg's Abandoned LLM and Neuro-symbolic Trading

Sunil: Sajjad, what's your opinion on this?

Sajjad: I don't know much about defense, but I do know a little about advanced financial systems — we were engineering some of those platforms long ago, so we have some experience there.

The most famous example in this space is Bloomberg. They tried to build an LLM on their own, spent tens of millions of dollars on it, and after a while gave up the effort and went back to using a frontier model.

They use frontier models largely for interpreting signals. At the end of the day, a trader is interested in all kinds of business and news signals, and to analyze those, an LLM is probably a good tool. But to replay trading strategies, they need symbolic AI — Monte Carlo methods, simulation and modeling, backtesting, options pricing. A good number of those are symbolic methods.

So I think it's a great example of neuro-symbolic systems being built by the flash trading crowd. It used to be all about speed at one point, but now it's about speed plus intelligence. Speed matters, but intelligence matters more — and intelligence here is a combination of LLMs and symbolic methods.

Even the LLMs here tend to be slow. These are not low-latency systems. So how do you generate candidate proposals, and then have your verifiers or Monte Carlo methods run simulations to see whether those candidates can stand the rigors of verification? I'm sure their proprietary trading strategies are worked out offline, the day before, and then they run a lot of simulations before pulling the trigger on whether to buy, sell, or short.

Those are examples of neuro-symbolic architectures, and they've evolved over quite a long period of time. So I can't speak to defense, but I can speak a little about what happens in the financial industry.

Wrap-Up: Converting Silent Errors into Caught Errors

Sunil: I think we've reached our limit. Anything else you want to add?

Sam: The argument is that rules are the durable asset. A better model makes the same verifier more valuable, not obsolete. Domain layers don't reduce model error — they convert silent errors into caught errors. They shift the risk from "the model might be wrong about anything" to "the rules might be incomplete." And that should be auditable, which is exactly our neuro-symbolic picture. So we're saying much the same thing Khosla is.

Sunil: That's it, I think. Thank you both — we'll talk later.

Sajjad: Thank you.