Episode 25 — A Four-Person Team and AI: Rebuilding Chip Design Around the Digital Twin

All opinions expressed are those of the individuals themselves, not necessarily of any company they work for.

Introducing Peter Suaris, Co-Founder of AxPro Semi

Sunil Baliga: Okay, welcome everyone, Token Drop, episode 25.

Sunil: Today we have a very special guest, Peter Suaris. I've known Peter for many years — he's the co-founder of AxPro Semi. Before that, just a wealth of experience in the semiconductor and software spaces — he was at Analog Inference, at Wave Computing (which acquired MIPS), at Cadence, at Atrenta, and elsewhere. I've really admired Peter, because he's not just a leader of teams — he's also very hands-on.

Sunil: Thank you for joining us, Peter. As always, opinions expressed are those of the individual, not necessarily the company they work for.

Sunil: One thing that's really caught my attention talking to you, Peter — you said AI has been a real boon to your company, and you've had to adopt new methodologies to boost productivity 10x or even more. What are these methodologies? What did you have to do?

Building a Cycle-Accurate Digital Twin of a Full SoC

Peter Suaris: Let me tell you the environment and the design challenges we had to tackle. At a very high level, we were building a hardware block that includes a CPU, an accelerator, and a whole subsystem — essentially, a full SoC.

Peter: When you build something like that, first you have to build a model of it — one that can actually run real-world designs. This is something people almost never do at a high architectural level, because you typically don't have a model capable of it. The model has to capture the caches, the disk I/O, and the functions of your accelerator and your CPU, all at a cycle-accurate level. Only when you model all three together do you actually discover the limitations of your hardware, and you can iterate.

Peter: To do that, you also need to run real-world designs — if you're running an LLM, you typically need at least a 7 to 13 billion parameter model, and we were able to do that. Only then do you find the real problems — you go to a large matrix, and you find you're wrapping around at the last column, you've lost everything in the cache, and you have to bring it all back in. That takes a hell of a lot of time. So you have to design techniques to optimize cache residency, and so on — these are things people simply don't find until they build a product and run real things against it.

Peter: If you have a good digital twin, you can forecast and predict all of that and iterate. It's important for the digital twin to be cycle-accurate, because then you can hand it to a customer, and they can run their own small designs well before the chips are even available — you can entice them, get their feedback, and good things happen from there.

How Chip Design Used to Work: Silos, Spreadsheets, and the Handoff Cliff

Peter: How has this been done historically? In the hardware space, it's been done in silos. The CPU architect defines things in the CPU. The SoC architect defines the cache subsystem. The kernel architect defines the other pieces. All done piecemeal, and the interaction between them is never actually very well designed or simulated.

Peter: Even those individual architectural pieces are built using spreadsheets, small models, and small programs. A CPU architect typically has a whole software team supporting him. Say you're doing a CPU with a branch prediction unit and want to tweak the algorithm — you tell the team what you want, they go build it, and a week later they come back with results. That loop repeats. It converges eventually, but you're never able to run a complete, interactive simulation of the whole real system.

Peter: Once that whole panoply of spreadsheets and block diagrams is done, a design team takes over. This is actually what happens today: you throw away all of that simulation work, start fresh, and the RTL team takes over until the RTL is done. The handoff kind of falls off a cliff right there.

The New Process: One Architect, AI, and No Separate Software Team

Peter: What we've done instead is take the architect — a hardware guy — and enable him to build this model without needing a software team, using AI. We trained him to use Codex. He simply prompts, and builds the whole thing himself — to the point where the entire iteration is done independently, no external team, nothing. He was very motivated to build this, so he ran all the experiments and all the tests himself. Just one person did what used to require a very large team.

Peter: Once that was done, we used AI again to build a digital twin out of it — again, simply by prompting. We could take the output, a C++ model, and generate a digital twin directly from it. Once we had the digital twin, we could boot Linux on it, run PyTorch, run LLM models, get results back, and feed them back to the architect, and iterate from there.

Peter: Once the digital twin was complete, we iterated again and refined it into a C++ model very close to hardware — all the pipelining, duplication, everything done by the engineer at that level — then refined further toward something close to Verilog. The last step, again using AI, converts it to RTL. That's the whole process.

Peter: This also enables something that used to be unthinkable: making an architectural change after you've gone all the way to RTL used to be prohibitively expensive — so many teams, so much testing effort. But because this whole pipeline is automated, we can make changes very fast, go back to the architect, and fine-tune things quickly.

Peter: This methodology isn't something most companies adopt — I know many startups don't even do this. But because of it, one architect and three people on the software team were able to build the whole thing. That's something that used to easily require a hundred engineers, even at a software company. It's a combination of methodology and AI that lets us converge much faster. But the key is the methodology — without it, you won't get there.

Why AI Is Only Powerful “In the Hands of the Expert”

Peter: The other thing about AI: it's very powerful in the hands of an expert. I would not put it in the hands of a novice, because you can go astray and not even know it. The architect who's built a CPU before knows when the AI says something wrong, or does something bad — he can correct it and move it along. That's a very key observation. Everyone on this team is an expert, very experienced in timing closure and verification — in the hands of people like that, it's extremely powerful, and it lets you get something working very quickly. That's been my experience.

Using the Digital Twin for Verification and Validation

Sajjad Khazipura: That's interesting — building a digital twin to verify and validate, through cycle-accurate simulation, how your eventual chip will behave, so design decisions that need to be made now, can be made now, rather than waiting to find out later. You've worked in the space of digital twins and formal verification techniques — do you only use formal verification, or do you do exhaustive verification?

Peter: In our case, we didn't use formal verification — it was exhaustive simulation, and we got AI to build the simulations too. The nice thing is, because we start from the digital twin level, we can capture the boundary vectors passing across module boundaries — when you're running a large LLM or whatever workload — and use those vectors throughout, even at the RTL level, to make sure the design is correct. Each module gets its input and output vectors at the periphery, derived from the top level, so you don't even need a large verification team. It's all conveyed top-down.

Sam Pooni: How does the digital twin incorporate things like cache misses, latency, traffic contention, memory contention, and packet loss?

Sajjad: There's a lot of stochastic behavior that happens statistically — are you modeling all of that exhaustively, or letting stochastics play their role?

Peter: They play their role, pretty much. If there's a cache miss, it goes to L2, gets put in a queue — exactly what happens in real hardware. It waits its turn, gets serviced, and if it's not found there, goes to the next level, and so on. There are some approximations — it's not exactly like hardware, there can be small differences in the cache and memory subsystem — but overall it's very close. It's able to predict all the nuances that matter for performance.

Building the Software Ecosystem Before the Hardware Is Even Done

Peter: Another important thing about the digital twin I haven't mentioned: when you're building an SoC like this, you also have to build a software ecosystem. If you're competing against other companies, you have to run your digital twin and say, “this is the performance we get.” But just building the digital twin doesn't give you that performance — you need a software toolchain that's genuinely efficient and can compete, when others have put hundreds of man-years into their own software. So you get to build and optimize the software toolchain at this stage too. The software gets done well before the hardware is done — that's a very important distinction, and it's what you get with the digital twin.

Injecting Branch Mispredictions and Boundary Tests with AI

Sajjad: Going back to branch misprediction — do you consciously inject certain mispredictions during a test cycle, or do you let events play out and hope mispredictions happen naturally, so you can exercise that functionality?

Peter: There should be boundary tests too — that's part of the architect's job. He knows what the boundary tests are, since he's the one who designed it. And with AI, you just say, “this is the condition I want to test” — you don't have to write the test yourself. It's very easy from that perspective.

Fault Injection Systems and the Difficulty of Running Them

Sam: But suppose, in traditional systems, we normally have fault injection systems — profiles you run through to validate different testing scenarios, where you can specify what kinds of failures you're looking for, across a whole set of failure modes. Any fault injection or emulation system. It's a terrific idea to use an emulator — I'm trying to understand what difficulties you ran into running that kind of system.

Sajjad: What Sam and I are both getting at, Peter: one part is deterministic functional testing. But there's a lot of dynamic, random, stochastic behavior too. How do you model and control that, without some kind of formal proving system? Since you said you're testing exhaustively — does that let you cover every case, or do you still need to depend on formal proving methods to avoid investing so much time and effort?

Peter: The testing we do at the architectural level is more driven by the application — you're stressing the limits of the hardware, so it's not an exhaustive test in that sense. Where you're talking about implementation-level correctness, that's where you'd need some formal verification techniques in addition to exhaustive simulation, if you can. But most companies I've worked at don't do that — they use exhaustive simulation, and you just have to spend a lot of time making sure you cover all the conditions properly.

Bringing Formal Methods and Domain Knowledge to Catch LLM Errors

Sajjad: Exactly — and that's really what we're trying to get to. We're in the software space, and what we're doing is helping customers use LLMs. As powerful as these tools are — and as you said, your CPU architect can detect when the AI makes mistakes and correct them — what we're trying to do is: in well-defined industrial domains, capture enough domain knowledge to catch errors as the LLM is making them, and then rectify them.

Sajjad: So we use exhaustive methods, as you correctly pointed out, and we also use formal techniques. If you can model your application domain using mathematical formalisms, you can avoid a lot of that laborious, exhaustive testing effort. That's the parallel we're trying to draw from the work you've done in the past — and are still doing now — to how it applies to what we're working on.

Peter: I haven't given it too much thought outside the design space, but the way I think about it: you come up with some sort of formal model, and through theorem-proving techniques, you prove that the output you've gotten conforms to that model. I think it should be possible even for LLMs — you have your initial assumptions, you have your model, and you have your outputs, and you make sure you're not throwing out something inconsistent with your concept of the model. It should be possible to come up with something like that.

Sajjad: That's exactly right — that's exactly the area we're working on.

Sam's Question: Can a Digital Twin Derive RTL, Not Just Validate It?

Sam: What Sajjad's alluding to, to put it very simply: can we use a digital twin not merely to validate RTL after it's written, but to derive the RTL architecture — select its precision and timing, generate its corner cases, and continuously correct it using measured silicon? Is that possible? We have a similar problem in our own domain, and we're trying to map it onto yours, since you've actually lived through it.

Sajjad: And Peter's been at this for a lot longer than we have.

Real Example: Building a 1GHz Floating-Point Multiply Tree on FPGA Fabric

Peter: I'll give you an example. Today I was working on building an FPGA fabric to run a matrix multiply — a multiply tree for floating points. This is the heart of LLMs — this is what you have to build, whether you're Nvidia or anyone else. This is the secret sauce; it's what makes each design unique.

Peter: We had to build this at 1GHz. Building something at 1GHz on a programmable fabric is extremely challenging, to say the least. When I started, I honestly thought this would never be possible with this methodology. But using C++, I was able to partition it and pipeline it. To get that high a frequency, you need to register nearly everything — just a few levels of logic between registers, over and over. Some designs — especially control units with loops — you can't do that with; you have to partition those into separate sets and manage the transfers between them. That architectural-level work had to be done iteratively.

Why Peter Still Insists on Manual Control Over Full Automation

Peter: But I prefer to do that partitioning manually, rather than letting AI do it automatically, because I want control. I'm afraid that if something goes wrong, I won't know what the hell it's actually doing. That's the real challenge with these automated correction techniques — even when it's possible. For example, when there's a timing problem, I give it the timing path and the source code, and it tells me which piece of code is causing that path, and what the fix is. Then I tell it to fix it, and it does — but I won't let it fix things automatically on its own, because I want to have control and understand what it's doing. It's a fine balance. You can let it loop and do a lot of things by itself, but in a huge system, if some little thing goes wrong somewhere, you have a big problem. So how do you manage that complexity? That's the real challenge with automated techniques.

The Parallel in Multi-Agent AI: Loop Engineering vs. Graph Engineering

Sajjad: In our parallel universe, we're running into similar problems. If you have multiple agents that are supposed to collaborate on a piece of work, the general conventional wisdom today is: give the LLM the objectives, the LLM gives the agents their objectives, the agents talk to the LLM to figure out what they need to do, and they try to achieve those objectives. And what we've seen is, invariably, they falter somewhere along the way. The answer to that has been: do it in a loop until you get it right — which means you're burning a lot of tokens. Some people call this loop engineering.

Sajjad: It's now evolving into something called graph engineering: you deterministically model the workflow — who needs to do what, and what the handoff points are. When you model that in a graph, deterministically, you significantly reduce the likelihood of errors, and you get more deterministic outcomes. That's happening more at the agentic level — the unstructured data the LLM processes and the answers it gives you. In your world, you have your CPU architect look at the answers and say, this is right, this isn't right — a human in the loop.

Autonomous Operations vs. Human-in-the-Loop in Business AI

Sajjad: Whereas the whole idea behind AI in business workflows is autonomous operation. If you have a human overlooking all of it, businesses push back and ask, then what's the point of the automation — why am I investing money in it?

Peter: But the productivity gain is huge, right?

Sajjad: That's true, that is indeed the key. And people are looking at fully autonomous operations — how do I run these processes lights-out, with agents taking over and running everything for me? And those agents can make mistakes, so we're looking at what oversight and correction mechanisms we need to inject — reviewing their work, validating whether it's right or wrong, and if it's not right, at least flagging it or getting the agent to redo that piece of work, so you eventually get it right.

Formal Verification, Explained for the Software World

Sunil: One thing I don't fully understand is formal verification, because my chip knowledge is dated — the only formal verification I knew was RTL-to-netlist synthesis verification, making sure those two matched. That's not quite what you're talking about here, is it?

Peter: Yeah, it's actually the same concept.

Sam: The one question we're fundamentally trying to answer is: what is the actual work of a symbolic verifier? When you counter-propose — can you teach it the thing it's trying to learn? Give it more use cases, generate thousands of them, so that when it proposes something, it's in line with what you want, and you've deliberately shrunk the probabilistic portion and made the deterministic portion big enough. That's the problem we're asking — can it be solved, and how?

Peter: Yeah, you're right — I don't know the answer to that, but that's a good problem.

Sunil: Next week's podcast, maybe.

Peter: That's utopia, though — if you can solve it, it's basically self-correcting. Gorgeous, if you can get there.

Formal Verification's Long History with Peter

Sajjad: You've been touching this space for quite some time — from your grad work, where I believe you found an alternative to simulated annealing, a somewhat more formal decomposition of the work to make sure it's done exhaustively and correctly, all the way to applying these verification techniques today, using both exhaustive and formal verification. We're drawing parallels and lessons from that world, and asking what we can learn from it — because software applications in the business world are designed to be deterministic. A money transfer from one bank account to another is a deterministic operation. Now you have probabilistic systems entering that world, so how do you control the quality of those probabilistic systems and filter out the errors? That's really the challenge.

Peter: Totally — I'm sure that's a very lucrative research topic right now, and a lot of people are working on it.

Sajjad: Correct. And that's where we are now — we're going back and opening up our math books to relearn formal verification. What is a theorem prover? What is a SAT solver? What are SMT solvers? What tools do you actually use?

Real Example: AWS's Automated Reasoning for Network Reachability

Sunil: How would you actually use those in the software world, Sajjad? I understand it in the chip world — RTL to netlist, you have something concrete to compare against. How do you compare in the software world?

Sajjad: Good example: Amazon has been using automated reasoning techniques across their cloud products and services. One use case is network reachability analysis. You have a VM, and around it, security groups defining what traffic can and can't touch it. That VM sits inside a VPC, and around the VPC you have firewalls — layers and layers of firewalls, networks, routers. To exhaustively demonstrate whether a given network packet can actually reach a VM or not, across all of those rules, requires formal proving. So rather than exhaustively testing every combination, they have a formal proving method that tells you, given these policies, whether the network is reachable or not — through formal methods. They've identified a lot of use cases that way.

Legal AI: Converting Contracts into Provable Assertions

Sajjad: I think now, if you've seen some of the legal AI startups — they're able to convert a lot of business contracts into assertions, and test one assertion against another through Boolean logic and theorem provers, to verify whether certain business clauses apply to you or not, and under what circumstances. Those are some use cases I can think of — I'm sure there are many more, but we haven't fully explored the space. We're really just beginning that journey now.

Security and Cryptocurrency: Why Concurrent Systems Need Theorem Proving

Peter: Another good area is security and cryptocurrency. Distributed computing — in cryptocurrency, everybody's running Bitcoin and smart contracts, everything distributed. Verification there is very important, and you can't just simulate it — theorem proving is really the way to go. You know about languages like Solidity — there's no way to test these properly in a distributed environment with so many parallel actors working concurrently. Anything can go wrong. The only way to handle that is by maintaining assertions and proving those assertions are never violated.

Wrap-Up: World Models, Feedback Loops, and Measured Evidence

Sunil: We're almost out of time — let's wrap up in the next couple of minutes.

Sajjad: Let me just sum up what Peter's been describing. Any business contract or obligation between two or more parties has certain rules. To demonstrate that every transaction stays within the bounds of those rules would normally require a lot of exhaustive testing. Can you instead verify and validate that those transactions are verifiably correct through formal methods — so you don't have to spend all that time, money, and effort exhaustively testing every boundary?

Sam: I think we actually touched on this already, Peter. From my perspective: a system of record supplies the exact RTL file, firmware, package, and calibration versions. The digital twin reconstructs the affected link and operating conditions. You use specialized solvers to evaluate signal integrity, thermal behavior, and control logic. This is where Sajjad's term “world models” comes in — comparing possible actions: retrain, change equalization, lower the lane rates, whatever it might be. A policy layer then selects only an authorized, safe action. The physical system applies the change, new telemetry gets written back as evidence, and the twin recalibrates if measured behavior differs from prediction. So what we're describing is a feedback loop running through all of that — and the real question is whether it actually works.

Sunil: I think we're out of time — thank you so much for joining us, Peter.

Peter: Thank you very much, it was a pleasure talking to you all.

Sunil: As always.