Episode 15 — The Invisible Layer: Why "Connective Tissue" — Not the Model — Is What Kills AI Projects at Scale

All opinions expressed are those of the individuals themselves, not necessarily of any company they work for.

Introducing Theo Forbath

Sunil Baliga: It’s Token Drop time, and today we have a special guest, Theo Forbath. He and Sajjad worked together for many years, and I was recently introduced to him through Sajjad. Theo is the founder of an organization called Above the Model. Before that, he was a Managing Director at Accenture and a Global VP at Cognizant — he’s very knowledgeable in AI.

When I was introduced to him, I read his Substack, Inside the Architecture: How Operating AI Actually Works, and I think that’s a great way to kick off our discussion today. Standard disclaimer: all opinions people voice here are their own, not necessarily those of the companies they work for, and we typically avoid discussing specific companies or products.

Theo, there was a line in your Substack I wanted to ask you about directly: you wrote about watching a proof-of-concept video “wow” executives, and then quietly stall six months later — and that the models are usually not the problem; the missing piece is the architecture. I didn’t quite understand what you meant by that. Can you unpack it?

The Missing Architecture: “Connective Tissue”

Theo Forbath: Sure. Working with my clients, what I’ve started to see is that they can successfully launch an AI pilot — that’s not the issue. In fact, more and more of my clients are taking some of those smaller, self-contained AI pilots and putting them into production at scale.

Where things fall down is an invisible layer — there’s no investment in what I’ve been calling the “connective tissue.” When companies haven’t thought through how to build shared context for agents being orchestrated, how to manage working memory and long-term memory, tool contracts, evaluation frameworks, and continuous observability — if you don’t put that in place from the beginning, going back and retrofitting your agents and infrastructure to support it gets very expensive.

What I’ve been emphasizing to clients is this: if you can’t demonstrate that each agent you build is cheaper than the last one, I’m not sure you’ve got your AI operating model right. That capability — being able to actually demonstrate that — is where the rubber meets the road for a lot of my clients right now.

Three Layers: Orchestration, Expert Agents, and Connective Tissue

Sunil: So the missing link is that AI operating system layer?

Theo: The connective tissue is one of three layers. First is the orchestration layer — the supervisory brain, if you will. It holds state across multi-step tasks, decides which specialist acts next, manages handoffs, and knows what to do when something goes wrong. Critically, it’s not an automated chain — it routes dynamically based not just on what it’s seen, but what it learns over time. If you can replace it with a switch statement or a sequential pipeline without losing functionality, it’s not real orchestration.

The second layer is the expert agents, or specialists — role-defined, domain-specialized, stateful, tool-calling actors that produce structured outputs with citations. A real expert agent, in my mind, is the model plus the architecture around it — architecture that both constrains it (managing hallucinations, which are real and aren’t going away) and equips it.

The third layer is the connective tissue — the piece nobody talks about and everybody underbuilds. It’s what lets you take a hallucination that happens on a single event and keep it from cascading downstream. That’s exactly what I see missing for a lot of my clients.

A Concrete Example: Insurance Claim Adjudication

Theo: One client, an insurer, ran into this directly with insurance claim adjudication. They put agentic agents in place with an orchestration layer. Adjudicating a claim depends on what the evidence shows, what the policy language says, what the loss facts are, what prior claims show, and what fraud signals are present — each of those is an independent expert agent.

If those agents hit an unexpected combination and it falls through to a default — which is really just a structured way of giving up — without real orchestration reasoning about that combination against the actual intent of adjudicating the claim, you see hallucinations start to cascade and affect the adjudication. That’s exactly why the orchestration and connective-tissue layers matter so much. And what I’m seeing across the board is that this layer gets treated as plumbing — assumed to exist in 2026, deferred, and built ad hoc when it’s built at all.

Two Analogies: A Company Without Shared Goals, and an Operating System

Sajjad Khazipura: I couldn’t relate to that more, Theo. I’d add two perspectives. First, a business one: you might have brilliant individuals, but if they’re not bound together by common objectives and a common operating model, it’s hard to form a company that actually meets its goals. You need defined roles, defined responsibilities, accountability, and exception-handling mechanisms — escalation paths, resolution mechanisms. For each of those functions, there are rule books, guidebooks, playbooks, runbooks describing who does what when something fails.

The second perspective is that of an operating system — I was an operating-system engineer for about ten to twelve years, and Sam worked at the same company. You can map this directly to OS concepts: a scheduler that dynamically assigns work across available CPUs so you keep making forward progress; specialized functions — I/O processors, graphics functions — that tasks get routed to when special capability is needed; continuous optimization for resource usage and throughput; failure reporting and recovery mechanisms, so you fail gracefully rather than abruptly; and handoffs that notify whoever’s responsible when forward progress stalls.

So whether you’re talking about it at the operating-system level or the organizational level, you’re really describing an operating system for a group of collaborative agents to work together — and that couldn’t be more relevant than it is right now. You can always build an individual agent or an individual function, but you only demonstrate real organizational productivity by getting agents to work together to automate whole workflows and show forward progress.

Why This Matters Even More in Agent-to-Agent Workflows

Sunil: It seems like what you’re describing, Theo, matters even more in a fully agentic workflow — agents talking to agents, with no human in the loop. I’ve mentioned before that a friend of mine running an AI startup told me his customers, if they get an answer they’re unsure of, just run the question again — they have the judgment to weigh what’s right. But how would an agent know to do that? It’ll take whatever answer it gets as gospel and move to the next step. That seems to make everything you’re describing even more critical in agent-to-agent workflows.

Sajjad: Theo mentioned evaluation frameworks — are you evaluating execution at each step, confirming a task actually met the requirements for a successful exit? And if not, what happens — do you restart it? Restarting has its own challenges. If I’m withdrawing money from your bank account, Sunil, and depositing it into Sam’s, and I fail after withdrawing but before depositing, you need atomicity — transactional consistency across the operation.

Theo: That’s right — observability and traceability across all the components, not per component.

Sajjad: Exactly. Frameworks like LangGraph are increasingly trying to build two-phase-commit protocols for exactly this — all-or-nothing semantics, so either the entire transaction completes or none of it does. Now extend that to distributed agents — an e-commerce buyer’s agents transacting with a supplier’s agents, running in two completely different environments and stacks, still needing to collaborate and make forward progress. That’s much, much harder.

A lot of people, as Theo said, don’t see immediate business value in building this kind of infrastructure — what’s the monetization of infrastructure alone? But it’s like building roads and bridges: you don’t monetize the road directly, but it’s what lets commerce flourish on top of it.

Five Diagnostic Questions to Detect a “Chain in Disguise”

Theo: I have five questions I use early with clients to figure out whether they really have an orchestration layer, or just a chain dressed up in agent vocabulary:

1. Can you replace it with a switch statement or a sequential pipeline without losing functionality? If yes, you don’t have orchestration — you have a workflow with agent vocabulary.
2. Are your agents stateful actors, or just wrapped LLM calls? Real expert agents make LLM calls too, but they also maintain memory within a task, call tools through defined contracts, and produce structured outputs with citations.
3. Does the connective tissue actually exist? Can they point to a real shared context layer, evaluation framework, and observability infrastructure — or is it “on the roadmap after launch”?
4. Can you trace a complete decision back to each of its inputs? For any output the system produced last week, can you reconstruct what context, which tool calls, and what reasoning produced it? If not, you don’t have the auditability that’s table stakes.
5. (implicit throughout) Is the system preemptive about failure, or purely reactive?

What Clients Are Actually Doing: Five Maturity Areas

Sajjad: Since you’re asking these questions of customers directly, Theo — what have you found? What are people actually doing?

Theo: People are all over the map on this journey. I look at five maturity areas companies need to focus on first:

1. Shared context — a single, authoritative, schema-governed, versioned representation of task state that every agent and the orchestrator reads and writes, with concurrency safety and role-scoped access.
2. Working memory and long-term memory as genuinely distinct things — working memory that clears on task completion, and long-term memory of patterns and prior cases that persists through governed write-back (what gets promoted, why, and with what provenance). Mature memory adds relevance-scored retrieval, expiry and forgetting policies, and poisoning defenses, so a single bad write doesn’t corrupt future reasoning.
3. Tool contracts.
4. Evaluation frameworks — continuous, automated testing at both the single-agent level and the end-to-end orchestrated level.
5. Observability — request-level tracing across all components, not per-component. For any decision an AI system produces, you should be able to reconstruct which agent, what context, which tool calls, what confidence, what payloads, and at what cost.

Enterprise Knowledge Fabric: Connecting Data Across Silos

Sajjad: It’s interesting you mention memory — we had a previous colleague of ours, Werner Goertz, on the podcast, and he talked about the idea of an enterprise-wide knowledge fabric. As agents discover, learn, and write to memory, are you extracting your company’s practices and knowledge from that, curating it, and building a real flywheel out of it? Since you’re constantly in front of some of the most advanced AI users out there, what have you seen?

Theo: Many of my clients are really internalizing the importance of getting the data right. Most are looking at a data-mesh strategy, sometimes using a data fabric from a vendor, but the real pattern is a central governance body for data architecture, taxonomy, and metadata, combined with a federated, fairly decentralized model — where each business unit curates and owns its own single source of truth (product owns PLM data, finance owns finance data, and so on), all adhering to a common set of data governance policies.

What AI really uncovers, honestly, is the politics that exist inside so many organizations — so much of that data lives in silos. The real value of AI shows up when you can cut across those silos: research understands what’s happening in sales, marketing can see what product management is doing to prepare the next generation of products based on what’s actually selling. That unlock comes down entirely to the quality of the underlying data.

Real Memory vs. Vector Similarity Search

Sam Pooni: One thing worth pushing on — Sajjad and I talked about this on a separate call too — is write-back heuristics and memory drift, and the tension between contamination and privacy. There’s also a real maturity gap here: vector databases give you similarity search, not memory. Real memory needs consolidation, contradiction resolution, and forgetting.

Theo: That’s exactly why I talk about working memory and long-term memory as two genuinely distinct types.

Sajjad: On the model-drift question — where I land is you have to baseline anything you’re going to ask a model or agent to do, across multiple dimensions, and put real guardrails on each piece of output.

Sunil: That’s it.

Sajjad: There will be drift in the nature of the data being fed to these models — that’s a foregone conclusion. Detecting meaningfully significant shifts in that data, and knowing when to retrain a model, swap to a different one, or add guardrails, takes real, ongoing effort. That falls under observability, but not the way most people think of it — a lot of people think observability just means watching flows, inferencing mechanics, and token usage. The more subtle piece is watching for data drift and model drift when you update to newer model versions, and being able to act on it.

Institutional Memory and the Knowledge Flywheel

Sajjad: Another angle here is institutional memory. AI gives us a real opportunity to capture knowledge that, as individuals, we always try to document and share with an organization — but we consistently fail at, for very human reasons. If you program these systems to store what they learn, in a disciplined way — harvesting and curating that knowledge and using it as a flywheel — that becomes a genuine business differentiator.

Theo: That connects to the test I mentioned: does the next agent you build cost less to build and put into production than the last one? That’s exactly what demonstrates you’re genuinely harvesting that information.

Sajjad: Exactly right. For those of us who’ve worked in data for decades, data quality has always been a struggle — and you’ve pointed to another real problem: data hoarding. If I own a dataset, should I share it with everyone, or only some people? But the actual value isn’t in who owns the data — it’s in how you fuse it and extract insight from it. When you’re talking about models, knowledge, and governance together, it’s really about distilling data into core nuggets of knowledge and integrating that knowledge — which is far more powerful, and much harder to do.

I think a lot of people are starting to recognize this shift — Google recently open-sourced a format for representing and modeling knowledge, which is a good signal of where the industry’s heading: from raw data toward distilled knowledge. The real question becomes how you help customers build flywheels that generate that knowledge, not just extract it from broad data sources — because agents and their transactions actually help build it. When knowledge is combined with action, you get real experience, and that experience feeds right back into the knowledge base.

Legal Liability and Multi-Agent Risk

Theo: The thing I think we all have to keep monitoring for is a model inventing a plausible-sounding reason for something. That reads as a UX problem on the surface, but what my clients are actually worried about is that it becomes a legal one.

Sajjad: Absolutely — we’re starting to see regulation and laws taking shape across different geographies. You can’t blame the model alone; you’re still the owner of that business or function, and you carry far more liability than the model does. Which means real investment in making sure these models are well-behaved — serving correct, timely, accurate information — matters enormously. And when you have multi-agent systems, with one agent feeding information to another with little or no human oversight, guaranteeing correctness and accuracy becomes significantly more critical.

Theo: I had a client who hadn’t invested in this invisible connective-tissue layer, and ended up with two agents flatly contradicting each other while answering the same client’s question.

Sajjad: That reminds me of a video I saw — someone had three different agents take turns counting from 1 to 100, and the sheer amount of coordination chatter between them dwarfed the actual forward progress. Genuinely funny to watch, but also a good illustration of the coordination overhead.

Theo: Right — and on the more serious end, you get a hallucinated wrong part number, or a wrong compliance spec for something like a nuclear power plant. That has real consequences. We have to make sure we’re putting the right guardrails back in.

Does Automation Quietly Erode the Expertise Pipeline?

Sam: Question for you, Theo — your architecture escalates to a human exactly when the AI is least confident, on the hardest, weirdest cases. But those routine calls the system now handles were how junior specialists used to actually learn the domain — troubleshooting, hands-on practice. Ten years from now, when your diagnostic agent escalates its toughest case, who’s actually left on the other end if they never got that training? Doesn’t that create a vacuum — is operating AI quietly consuming the expertise pipeline that the whole human-in-the-loop safety story depends on? We did something similar with manufacturing: we offshored it to China, and now we’re having a very hard time bringing it back. Do you think something similar could happen in AI, once junior specialists are largely replaced by agents?

Theo: I think junior specialists are going to look very different, not disappear. There’s going to be a new category of employee — I think of them as “the builder”: someone trained in a domain (pharmacy, life sciences, whatever the field is) who’s also genuinely comfortable writing Python code and working with machine-learning tools — not just comfortable with AI as a buzzword. I think that’s who the next generation of college graduates who become most valuable will be, and that’s who junior roles morph into — the people who actually drive AI adoption at scale over time.

Why Understanding “Under the Hood” Still Matters

Sunil: Why would they need to know Python specifically, though? The conventional wisdom with today’s AI coding tools is that you don’t need to know any language — you just describe what you want and run the code.

Theo: I think it’s like the early days of programming — did you need to understand how memory worked at the CPU level? Eventually that got abstracted away enough that nobody thinks about it anymore. But I think we’re still in the early innings with AI — really the first innings, on a ten-year time frame. What gets us further is a deeper understanding of what’s actually going on under the hood, rather than treating AI like a black box. I think that’s exactly where we’ll get into trouble in ten years if we get too comfortable treating it as one.

Sajjad: Do your customers actually voice these concerns to you?

Theo: They do — and it’s causing a real shift in how they approach their AI programs. They’re getting less comfortable simply asking outside partners to build this for them, and increasingly see it as critical to train their own people to become genuinely AI-first thinkers — not just applying AI to an existing end-to-end process, but reimagining that process as AI-driven from the ground up. They don’t feel they can fully trust partners to do that reimagining for them.

Sajjad: Exactly — I think that knowledge-creation frontier is going to be critical, because AI can recall from memory every pattern it’s already seen, but it hasn’t reached the ability to create genuinely net-new knowledge. That’s where humans can keep doubling down — creating net-new knowledge across any domain, whether it’s biology, art, architecture, or anything else.

How Does an Orchestrator Know When to Give Up and Escalate?

Sam: One last question, Theo — this has been on my mind for a while. The orchestrator piece is fundamentally a two-way street: when it can’t reach a conclusion, it hands off to a human in the loop. That’s a really important point, but it’s also, from my perspective, one of the hardest parts to actually design — especially at the orchestration layer. How does it know when to give up?

Theo: Right — how does it know when something’s actually going wrong? That comes back to the guardrails point I made earlier: going into any large-scale, multi-step agentic task, you need to have done the baselining in advance. You get early signals, and you maintain continuous observability within the connective tissue specifically watching for drift in an agent’s or model’s output. It’s not a reactive approach — it’s a preemptive one, for managing an end-to-end process that’s meant to be AI-first and human-second.

Wrap-Up

Sunil: I’m afraid that’s going to have to be the last word on this one. Thank you so much, Theo — I’ll put a link to your Substack in the episode description so people can read your articles themselves; it’s genuinely interesting. Thank you for joining us today, really appreciate it.

Theo: I really enjoyed it — good to meet you, Sam, and thank you both.

Sajjad: Thank you.