Episode 8 — Forward-Deployed Engineers: Why AI's Hottest New Role Is an Old Idea Wearing New Clothes

Introducing Satya Mantha and today's topic

Sunil Baliga: Today we have Satya Mantha, a key architect at Fission Labs, a leading AI services company I know well — I used to work there myself, years ago. One thing I've seen a lot of buzz around lately is the concept of forward-deployed engineers. Some of the major model companies are leaning heavily into this — partnering with services firms, and in at least one case, I recall Anthropic partnering with a PE firm to help stand up a dedicated services organization. I thought it'd be interesting to dig into the concept of forward-deployed engineers today.

There's a lot of buzz about this term, and when I first heard it a year or so ago, it reminded me of my own time in the services industry, where the equivalent role was simply called an "on-site engineer." Is there really a meaningful difference between a forward-deployed engineer and an on-site engineer? And why is this concept having such a moment specifically in AI? Satya, I'll kick it to you first.

Satya Mantha: Thanks for having me, Sunil — great to be here. To really understand the concept of the forward-deployed engineer, it helps to start with a bit of history. The term was actually coined by Palantir back in the 2010s. They faced a very specific challenge: they primarily worked with government agencies and military organizations, where data sets came with strict compliance and regulatory requirements. Their products couldn't simply be sold and shipped off-the-shelf — there were real differences between environments that had to be worked through directly.

So Palantir started building teams of on-site people who were deployed directly within those client environments. Those engineers would learn the client's specific datasets, learn the classified systems they were working within, and effectively bridge the gap between the core product and what implementation actually required — building data pipelines, doing hands-on implementation work. It was really field engineering.

Over time, they started feeding what they learned at each client site back into the core product itself, creating a continuous loop of learning and product evolution paired with deployment engineering. As I researched this further, I found Palantir had a fairly distinctive organizational setup: two types of teams, an "Echo" team and a "Delta" team. The Echo team was made up of domain experts constantly studying the implementation domain of the product, identifying the highest-value use cases for the customer. The Delta team was the fast-moving engineering team that actually built custom solutions on top of the product for that specific use case. Learnings from that work would eventually get transferred back into the core product team, who'd architect it for scale and turn it into a real out-of-the-box product. That's the history — and that's how the term "forward-deployed engineer" gradually emerged, since these engineers are literally working forward, out in the customer's own environment.

What actually makes an FDE different from a traditional on-site engineer

Sunil: Sajjad, given your own services background, what's your read on this?

Sajjad Khazipura: I think there's one genuinely unique distinction here. Forward-deployed engineers still have direct access to the core product codebase, and they're enhancing that product at very high velocity, directly in the context of the specific customer problem they're solving. They're empowered to make changes rapidly to create a genuinely good fit for that customer.

The obvious downside: when you have multiple forward-deployed engineers working across different customers, each customizing the product for their specific environment, you can end up accumulating significant technical debt on the back end — changes to the codebase that aren't necessarily compatible with each other. So this really requires senior, mature people — not just "any engineer you deploy." These need to be people with genuine product vision, who understand the product deeply and function almost like product owners. They bring real consulting skills, but they're hands-on engineers — not just on-site coordinators handling reporting, management, and orchestration. They actually write code and deeply understand the product. That's the key nuance distinguishing this from the traditional consulting on-site engineer role.

Sunil: Sam, looks like you've got something to add.

Sam Pooni: In my early years I was at HP Research Labs, then did a few startups, and eventually became head of professional services for the Americas at a previous company — coordinating with clients, talking to customers and the C-suite, and bridging the gap between the CDO's office and pre-sales. One of the most interesting shifts I saw, especially toward the end of that role, was the AI industry's rapid move toward larger, smarter, faster, and increasingly multimodal models.

What we're really seeing now are challenges that are genuinely different from what we faced in traditional professional services, even when we were standing up massive data centers for customers. The real challenge today is deployment, orchestration, governance, observability, scalability, infrastructure integration, and operational reliability. So the natural question becomes: what actually is a forward-deployed engineer? It's a genuinely hybrid role that's emerged very recently, blending software engineering, systems engineering, infrastructure expertise, architecture, and custom integration all together.

Unlike a traditional software engineer, FDEs deploy systems directly into production, solve infrastructure problems, integrate with the customer's existing ecosystem, automate workflows, optimize deployments, and often bridge product engineering with real customer operations. FDEs matter more right now specifically because modern AI systems are distributed, infrastructure-heavy, GPU-intensive, network-intensive, and operationally complex. The question is no longer "can the model work" — Sajjad's said this many times — it's: can it scale? Can it integrate? Can it be governed? Can it operate reliably? Can it fit into real enterprise workflows? Can it run cost-effectively? Enterprises consistently underestimate infrastructure complexity, deployment effort, data quality challenges, governance requirements, and operational costs — which is exactly why so many AI projects stall out right after the demo phase. Satya can probably speak to how common that is.

Why now? Echoes of the big data era

Sunil: I want to push on something — every challenge you two have described, I've heard almost verbatim in conversations about big data and ETL pipelines years ago. What's actually new about AI that's brought this concept back into the spotlight? Or is it really just a rebrand of something that's always existed?

Sajjad: I can speak to that. When people first started talking seriously about big data — around 2012, 2013 — I was at a consulting company engaged with many different customers across industry sectors. A lot of people called that period "POC hell," and it lasted close to four or five years. It wasn't just us — plenty of peer consulting firms, and even product companies themselves, went through the same thing, because everyone was kicking the tires: every customer was running POCs, validating ideas, and then having to go seek funding for actual production deployment. Along the way, there were always real questions around security, compliance, and data residency — so it took years before that industry genuinely matured.

In some sense, I'm glad the industry is now acknowledging the FDE role explicitly. In the past, systems integrators effectively played this role on behalf of product companies, helping them build and deploy solutions at scale inside enterprise IT organizations. Now, product companies themselves have stepped up and recognized the importance of this function directly — trying to short-circuit the cycle and reduce time to production. Otherwise, you end up spending four or five years in what people used to call "production hell." That's really one of the key insights I've taken from having lived through this before, watching it play out again in the AI product industry.

Sunil: Satya, what's your take?

Satya: I'd echo what Sajjad said. My own background includes ERP implementation and product work — and the same pattern applies there. ERP companies like Microsoft and SAP, along with other major platform vendors, build the core product. Then you have product engineers, systems engineers, and systems integrators — we were an ISV at one point, building on top of the core Microsoft ecosystem — plus implementation teams and consultants who'd go on-site to implement the product at specific customer locations. That creates real layers of software and product complexity, and it increases the difficulty of deployment and implementation given each customer's specific needs and constraints. In my view, the concept of the forward-deployed engineer really stems from exactly this kind of experience across many prior technology waves.

From systems of record to systems of intelligence

Satya: In AI specifically, as Sam mentioned, new challenges are emerging. We're moving from implementing "systems of record" to building "systems of intelligence." When you're building a system of intelligence, you're dealing with LLMs that hallucinate — which raises real questions: how do you do context engineering against a client's specific data sets? How do you build new knowledge bases? Which vector stores do you need? What hyperparameters matter for evaluation? What does the RAG pipeline actually need to look like, and how do you evaluate whether the model is hallucinating versus genuinely grounded in the institution's actual core knowledge base?

These concerns are real because foundation model companies publish benchmarks based on open, global datasets and can legitimately claim, say, 80–90% accuracy on coding or other tasks — but that tells you very little about how the model will actually behave once it's implemented against your specific data, inside your specific organization, for your specific business processes. That's exactly why these skill sets — forward-deployed engineers — are so necessary.

Sajjad: Couldn't agree more. A lot of these models are trained primarily on internet data, but once they meet real enterprise data, their behavior becomes genuinely unpredictable — you should expect surprises. That's a core reason you need forward-deployed engineers: to actually adapt the product to the customer's specific needs. None of the customer's existing technology stack is going away anytime soon either — you're simply adding a new layer on top of an already deep stack of legacy technology. This new layer needs to fit in, work with, integrate, and correctly infer against the customer's existing data and systems. That complexity isn't disappearing quickly.

Why model companies are building their own services arms

Sunil: I think that answers another question I had — why the big LLM companies are increasingly partnering with, or even standing up, their own systems-integrator organizations.

Sajjad: Right — some of them are opening their own professional services organizations entirely.

Sunil: Like Anthropic.

Sajjad: Exactly — we've heard about that from Anthropic, and there's been press coverage of OpenAI doing something similar with a deployment-focused company as well. They're building something closer to a McKinsey-style consulting and engineering organization, combining business consulting, technology advisory, and actual hands-on engineering and integration work. I'd guess they see this as a way to accelerate product deployment and help customers reach their goals faster.

Part of this also comes down to scale — compared to a typical ERP deployment or another standard application deployment, AI has such a broad cognitive surface. You're taking that huge capability surface and trying to make it work within a very narrow, specific problem domain.

Sunil: I don't quite follow — you're saying you take a large cognitive surface and apply it to a narrow domain?

Sajjad: Right — take a frontier model. Its capability surface is enormous; it can do an incredible range of things. But when you're working inside an enterprise, you're applying it to one specific business process, one narrow operational domain. The question becomes: nobody actually knows how that model will behave in such a narrow, specific context. First, the model's been trained largely on internet data, not enterprise data — and enterprise data is inherently more specialized and proprietary to that specific company. Business processes and workflows are even more unique still. So the level of specificity and uniqueness goes up dramatically, while you're working with a comparatively generic, generally-trained frontier model trying to operate in that narrow space. Nobody knows in advance exactly how it'll behave — which is why you need genuine last-mile adaptation work, done by people who are empowered to understand what the customer actually needs and who can make real product-level changes to fit that specific domain. That's a genuinely unique, increasingly in-demand skill.

Sam: Are you describing something like supervised fine-tuning specifically?

Sajjad: It's broader than just the model itself — there's RAG, increasingly knowledge graphs, security considerations, and a lot more adaptation work required beyond model tuning. And then there's data locality — even in the big data era, standing up a proper data lake wasn't easy. A data lake was a genuinely broad surface — more of a platform than a single point application, serving many different use cases at once. That breadth raises a natural economic question: who actually funds it? Usually the people funding it have a specific, narrow business problem to solve, and you're deploying a much larger platform investment to solve that one point problem — even though that same platform could serve many other use cases too. If you're a process owner, the natural question is: why should I invest in such a large estate when my needs are so narrow? Can those costs be amortized across other lines of business?

I'm fairly confident those are exactly the kinds of questions organizations will need to work through as they deploy AI — because while the upfront deployment cost can be steep (building GPU capacity, data pipelines, connecting to data lakes, building out security), all to solve one specific use case, the real argument becomes whether those costs can be amortized across other business processes, teams, and departments. Should everyone pay their fair share, or should this be a centrally funded initiative that benefits the whole organization? You genuinely need consulting and advisory skills — people who can run a real cost-benefit analysis — as table stakes for deploying this kind of AI infrastructure today.

How long do forward-deployed engineers actually stay engaged?

Sunil: One question for you, Satya — how long do forward-deployed engineers typically stay engaged with a customer? Do they come in at the start of a project to help architect the solution and then move on, or are they involved throughout?

Satya: In my experience, it's a continuous process. Once you deploy a solution, you still need to manage its ongoing sustainment — going back regularly to look at what's been implemented and continuously improve it. With AI models specifically, you can see progressive degradation over time, because the surrounding context keeps changing and the underlying knowledge keeps evolving. That's exactly why, as Sajjad rightly pointed out, this isn't a one-and-done engagement — we're dealing with a fundamentally stochastic system.

You need to make sure context engineering is handled properly on an ongoing basis, and that really can only happen from within the customer's own environment. Context engineering itself involves a whole range of activities — a lot of it rooted in data engineering, since data lakes remain the primary source of knowledge driving these organizations. Beyond that, you need to continuously ensure the context being fed to the LLM for its final output is evaluated and monitored on an ongoing basis for accuracy and groundedness. So in a nutshell, this is a continuous activity — and that's exactly where a lot of the real value in this space comes from.

The sheer technical breadth required

Sam: I think what Satya's getting at is that today's real deployment problems look very different from traditional deployment work — Kubernetes integration issues, GPU driver problems, network bottlenecks, storage performance issues, model-serving optimization, inference scaling, multi-cloud integration (AWS, Azure, and so on), identity integration, compliance restrictions, deployment automation — the list goes on. As Sajjad said, the surface area here is genuinely very broad.

That raises a real question about specialization. Historically we talked about SREs as domain experts in one specific area. Here, the domain itself is vast — Kubernetes, GPUs, networking, and everything in between. It really looks like an entire technology stack, from Layer 1 up through Layer 7, specifically for AI. You need real specialization at every level: the chip level, the GPU driver level, optimization at the PyTorch level with custom kernels, the applications built on top of that, and then, above all of it, the fast-growing agentic layer and multi-agent communication, which is its own rapidly expanding domain. This isn't something you can neatly draw a boundary around and shrink down. It's a genuinely vast space, and it requires the full lifecycle — day zero, day one, day two — continuously refined based on real experience, exactly as Satya described.

That's exactly why the forward-deployed engineer role becomes so critical — they deeply understand the systems involved. In typical enterprise engagements, you rarely get pulled off an account once you've built up years of deep experience with that specific customer. Now imagine someone specifically optimizing the AI stack for that customer over an extended period — they become genuinely invaluable, because they understand every metric and every moving part that goes into that customer's AI deployment.

Wrap-up

Sunil: Sam, I think we're out of time. Satya, thank you so much for joining us — our special guest star this week, so to speak. Really great conversation — thanks again, and we'll talk to you all soon.

Satya: Thank you, Sunil — great being here.

Sajjad: Same here, guys.