Episode 22 — Why AI Agents Forget: Graph Databases and the Agent Memory Problem

All opinions expressed are those of the individuals themselves, not necessarily of any company they work for.

The Token Drop backstory & meet Arun Sharma

Sunil Baliga: Okay, so — Token Drop. We have a special guest today, Arun Sharma, from LadybugDB. Before I get into his background, I wanted to take a moment to talk about the backstory of the podcast, because a couple of people reached out and said, “I enjoy your podcast, but what's the backstory? How did you guys get started?” I don't think we've talked about that before — maybe in the first episode — so let me take a minute.

Sunil: We got started as a company last year. We felt the need in the marketplace was for trustworthy, reliable AI, and that AI — LLMs — need some help to achieve that. An LLM can use help in the form of graphs, reasoners, verifiers, and cartridges to inject domain knowledge and company knowledge, to drive hallucinations towards zero. So we got started last year along that path. In our internal discussions, I used to enjoy listening to Sajjad and Sam talk about various AI topics, and I felt that maybe other people might enjoy it too. Those conversations were unscripted, so we just continued that, and it's evolved to where we are today. That's the backstory for the folks who were interested.

Sunil: So let me talk a little about Arun. I actually met Arun a bunch of months ago at a wedding — a fabulous, fun wedding — and I was happy to meet him there. I lost track of him, and I was delighted when he reached out saying, “Is that you on the Kurt Cagle podcast?” I was happy that somebody recognized me. We met up for coffee, and I understood that he's now doing something in the graph database space — a perfect area for our podcast. Interestingly, I saw you were also a Linux kernel committer back in the day at Facebook, so offline I'd love to ask you about that; it seems like a really cool project.

Sunil: So we'll get started. As always, all opinions are those of the individual, and not necessarily of any company they work for. Arun, it was great to talk to you and learn about Ladybug. Maybe you can tell me about it — how it fits in the evolution of graph databases, where it fits, and why — and we can kick off our discussion that way.

What is LadybugDB? Graph databases, NoSQL & the relational debate

Arun Sharma: Great. LadybugDB is an embedded, columnar graph database. Graph databases have been around forever, but people really started noticing them about 20 years ago. They were seen as part of the NoSQL movement — if you go to the Wikipedia page on graph databases, it describes them as a NoSQL database. That's a controversial statement to make in 2026, if you're looking for controversies, because there's a significant contingent of people who believe graphs are a natural evolution from relational databases: you take relational databases, make six-way joins super efficient, and you get a graph database. That's the other view of what graph databases are.

Arun: Over these years we've had the semantic web, RDF, and label property graphs — the two big subdivisions of the graph database community. My personal view is that they're not in conflict; they're perfectly complementary. You can build semantic databases on top of label property graphs if you make them work together.

Ladybug Memory: why one storage engine can't solve agent memory

Sajjad Khazipura: Fantastic — thanks for that backdrop, Arun. You also have a project called Ladybug Memory. Can you speak to that, and how it relates to LadybugDB, the database?

Arun: Right. LadybugDB is an open-source database — a project where I work with collaborators from around the world. The database started its life at the University of Waterloo in Canada; it was formerly called KuzuDB. KuzuDB was acquired by Apple sometime last year, and there were a bunch of forks. LadybugDB is one of them — but it's not a company. Ladybug Memory is a company that supports the LadybugDB project.

Arun: What it seeks to do is build an ecosystem of projects. You need a graph database, but a graph database alone — or any single storage engine alone — is insufficient to solve the agent memory problem. That's the primary thesis. If you say, “I need a row-oriented store, a column-oriented store for analytical queries, and a graph store,” and maybe a graph-analytics piece — a common example is people using SQLite and NetworkX, a common combination among data scientists and practitioners — then how do you put those multiple pieces together and make them work as a single, coherent unit? That's the problem Ladybug Memory is trying to solve.

Inside the engine: DuckDB's influence and the MMAP surprise

Sajjad: And when you say agentic memory — you map the database into memory, so the database objects are now available as in-memory objects that can be updated transactionally and read with low latency, thereby enabling responsive agentic AI workflows. Is that correct?

Arun: That's right. If you look inside LadybugDB, it looks a lot like DuckDB — DuckDB's fingerprints are everywhere. The buffer manager, or buffer cache, works like any other relational DB's buffer cache: there's a notion of pages on disk and pages in memory, and when you write, the data goes to a write-ahead log, and then a checkpoint writes the write-ahead log to the disk pages. So the architecture isn't very different from other databases.

Arun: The one thing that surprised me when I looked at Kuzu — which became Ladybug — is that most database people don't like MMAP. They think the kernel people are crazy, and that the OS doesn't really know which pages to cache and which not to. There's a belief in the database community that if you're using MMAP to manage your buffer, you're an outlier — database people don't do that. But Kuzu has that design. So if you start up a LadybugDB instance and look at the process size, even if you have 100 rows in your database, the process size will be 8 TB, because it maps this huge amount of virtual space and pages things in and out of it. So it has a unique design in how it manages memory, but much of it is conventional.

MMAP war stories: dynamic linkers, ELF & 64-bit vi

Sajjad: That brings back fond memories. I started working on the MMAP system call back in 1990 — I'm dating myself here. We had a small company called Lynx Real-Time Systems, and we were doing some work with them. One of the projects was to design a dynamic linker. AT&T SVR4 had just come about, with dynamic linking and the new ELF — Executable and Linkable Format — object files. We were trying to study how the ELF format would work, and it turned out all those segments in the file were mappable to memory: your text, your read-only data, even the stack — the entire process address space was constructed from MMAP system calls.

Sajjad: That's when I realized — going through the experimentation and eventually building a dynamic linker, which I stuck with for many years — that as simplistic as an MMAP call looks, you need to understand the semantics of every parameter. Get one parameter wrong and you could have segmentation violations. It's a really powerful system call, but you've got to use it carefully. In fact, at one point I had to use it when working on the vi command-line tool. It was running out of memory for large files — we were adding large-file support, expanding the file offset size from 32 bits to 64 bits. Every command-line tool and library that dealt with files had to handle 64-bit offsets. We found the memory allocated to vi from the heap was inadequate, so we had to map memory from dynamic memory segments. Once again, MMAP came to our aid. It's a fascinating system call — use it carefully, but it's really powerful.

Sunil: Sam looks like he wants to say something. Sam, you look like you're being held back — you really want to say something.

Sam Pooni: No — everybody had bruises with MMAP, so it's not like… MMAP is a terribly simple call when you look at it, but the parameters, as you said, you have to be very careful about — there are things about memory protection, reads, and writes that you need to know. During my days I was very eager; I wanted to try different things, and it was a perfect ground to work on MMAP. We thought we were pretty cool in those days. But later I found out how important that call was for memory mapping. The beauty of it is — when I started working on device drivers, there are two types: you can treat a device as an I/O device, or you can map the whole thing into memory and operate on the register space. It was interesting to see the different ways of doing accesses to devices — memory-mapped at that level, or as I/O. We read about these in operating-system classes, but when you're really working on code, you know the big difference.

Does “embedded” mean your knowledge has to be small?

Sam: But I have a specific question for Arun. I'm changing the topic a little. Embedded means the graphs must fit on one machine, right? That's a bet that useful knowledge is small. Is that true?

Arun: Great question. This goes back to my days at Google, and getting used to life outside of Google. One of the jokes was: if a Google engineer leaves Google and goes to a startup, and they're looking to build a 2 GB database, they say, “I can't — I need Spanner, or I need Bigtable, one of these distributed systems.” But the majority of problems don't need that kind of scale. That's the core idea behind DuckDB, and it's supported by market research: even for OLAP, a majority of workloads out there fit in about 100 gigs. That's within reach of a conventional laptop these days. They took a MacBook from 2012 and made demos showing how DuckDB can run really large workloads on consumer-class hardware.

Modular pieces: partitioned tables & a Neo4j (Bolt) wrapper

Arun: Having said that, there are workloads — the world's knowledge is not going to fit on one laptop. But the idea behind Ladybug, and a number of other projects, is to build modular pieces: an embedded database, a network protocol, a load balancer — all generic pieces — and then you combine them into a more powerful distributed system. There's work going on to do exactly that. The release landing in two days has support for partitioned tables: you can create a table partitioned by hash or by the value of a column. We don't implement range partitioning yet, and it's still not distributed — it just creates subgraphs on the same machine — but the idea is you can use this framework and write a wrapper around it. One example: Bolt is a network protocol that Neo4j uses, and there's a Bolt wrapper around Ladybug. So if you're a Neo4j user, you can run this Docker container and talk to Ladybug as if it were Neo4j. That's one example of developing an embedded database and a network protocol separately. Similarly, the distributed goodness — a load balancer, sharding, write-ahead logs — can all be built as a support structure around a core database engine.

Why columnar instead of RocksDB / LSM

Sajjad: A lot of people use RocksDB under the covers, and you're not — you're writing directly to a columnar file format. Why is that? Why do you choose to do that?

Arun: My history with RocksDB goes back 10 years. RocksDB is backed by an LSM — a Log-Structured Merge tree. The core idea there is that if you have many small writes coming in, the LSM collects and coalesces them into bigger units that go down a tree when compaction happens. It's good if you're getting many small writes and also have a significant read workload — that's when you'd use an LSM. The idea behind DuckDB and LadybugDB is that they're engineered for read-dominant workloads. So if you have a graph that can take writes, but reads dominate over writes, that's where columnar engines come into the picture. That brings up the question of what to do about the writes — that's a whole new discussion — but I'm curious where you want to take it.

What Ladybug Memory actually is

Sajjad: No, thanks for explaining. I just want to correct myself, for the audience's sake. I thought Ladybug Memory and LadybugDB were two separate entities. They are two different entities, but Ladybug Memory is not a product in its own right — when you refer to in-memory access, you're still talking about an embedded database; it's just mapping those database objects from the object store into memory. And because it's in memory, you now have low-latency access, and that's how you enable those agentic workflows. Is that now correct?

Arun: Ladybug Memory is very early in its lifetime, so it's still getting developed. You can think of it as an agent-memory solution that uses LadybugDB as its storage engine. There's a very simple implementation: for example, if you have Claude Code or your coding agent and you want to add a persistent-memory feature to it, you could use the simple implementation available on the Ladybug Memory GitHub. There's a Hermes agent plugin to demonstrate the concept, and LangChain and a few other plugins are possible. But the actual product, still under development, is a more holistic approach — multiple storage engines, replication between them, consistency — solving a real business issue: my LLM forgets, it hallucinates, how do I make it work better? I was happy to hear the discussion on this podcast, where you talked about many of the same topics — trustworthy AI, something that doesn't hallucinate.

A trillion dollars in, still no determinism in the AI stack

Sajjad: Yes. I read that blog post you pointed us to. We're in agreement there — there's a serious problem to be addressed. After close to a trillion dollars invested, we still don't have determinism to show in the current AI stack, so something needs to be done. That brings us to a question: what kinds of applications have you seen people wanting to build over LadybugDB? Each time I talk to a customer, it's a new domain, a new use case. Just yesterday morning, Sunil and I met with someone narrating customer requirements — one set from a bank, another from a telco vendor, another from a different domain altogether. The diversity of requirements is growing, and people are realizing you need systems of this kind. I was curious what your experiences have been.

Real use cases: code knowledge graphs, Uber's token costs & Git Nexus

Arun: The success LadybugDB has had so far is primarily in code knowledge graphs. The use case: companies like Uber find that engineers are spending too much money on tokens, because the engine uses very primitive tools like grep over and over on the codebase, costing a lot of tokens. So a number of companies build knowledge graphs — they go through the code, find the modules and directory structure, and create higher-level knowledge to persist in a database. Some use Markdown files, JSON files, sometimes SQLite — and then they run into the limitations of each of those systems, and eventually discover LadybugDB. It can do vector search, full-text search, and graph search — and, more importantly, the graph-vector hybrid, which I think is the one people desire most: part of the query is graph, the other part is a fuzzy vector search, and how do you do it efficiently? Something launched on Product Hunt yesterday — a project called Git Nexus, built on LadybugDB. And there are people who were formerly using KuzuDB; now that KuzuDB is no longer around, they're naturally gravitating toward LadybugDB. Code is a big use case, but there are others — someone is using Ladybug to parse SEC filings, the EDGAR reports, the finance data, and saving that in Ladybug; that's a customer story on the website.

Sajjad: That reminds me — we were running an HPC practice long ago, and one of the hyperscalers asked us to parse all the EDGAR reports. Those are in XBRL format. We extracted information from the XBRLs and provided a ready-to-query interface so you could query corporate actions, company performance, quarterly reports, and such.

Scaling past the laptop: Graph Lake, the IceBug format & Hugging Face

Sunil: I was going to ask — Arun said most of the data can be stored in about 100 gigs. What happens for some big enterprise applications who say, “That's nothing”? Do you tell them to go use somebody else, or do you have a solution to support them?

Arun: Great question. Like Sajjad was suggesting — DuckDB started as embedded, and then they're doing DuckLake. The Ladybug equivalent is called Graph Lake — I ran a poll, and that was the name people appreciated the most. What is a “graph lake”? Instead of saving the data in LadybugDB's native columnar storage, which is a single-file database like SQLite, you store these files as Parquet files on an object store. And it's not just a Parquet file — there's some more structure specified, and the name of this format is the IceBug format. I'm trying to work with the ecosystem — if there are other graph database vendors interested in adopting this format, I'm more than happy to work with them. It's an open format; there's nothing proprietary about it. The good thing is the Ladybug CLI can query this without you having to do anything. It supports well-known protocols — HTTPS, and something called XET, a protocol supported by Hugging Face. So we host some knowledge graphs on Hugging Face, and you can query them from the Ladybug CLI without downloading — there's no ETL, no ingest. This is a technology demonstration. There are also existing data-lake formats such as Iceberg and Delta Lake / Unity Catalog — there are extensions, probably about 15 on the Ladybug website — and it's possible to connect to your Snowflake or Databricks and use the graph query engine on external storage as well.

Does adding time blow up a knowledge graph?

Sunil: When we talked to Kurt a couple of weeks ago, we discussed how incorporating time — knowledge graphs are really not designed to track time. Is that a situation for you too?

Arun: No, it isn't. We have a very generic data model — it works exactly like DuckDB. You can create a node table, which… DuckDB doesn't use the keyword “node” — for them, everything is a table. The CREATE NODE TABLE in LadybugDB works exactly like DuckDB, but it also has a second type of table called a relationship table, which works more or less like a many-to-many table in a SQL database. And we have no restrictions on how many columns you can have.

Sunil: But when you start incorporating time, doesn't that blow up your database? Time is always moving, so wouldn't you just have a ton of time in your graph?

Arun: Great question — what happens to storage when you add the time column?

Sunil: By the way, you didn't pay me for any of these; I'm just asking them.

Arun: So, if you have a billion edges in your graph, and let's say only 1% of them actually have the time — 99% don't — the question is, are you paying the “time tax” for the 99%? The answer is no, because we use the columnar format on disk. The way columnar works, the time column is stored separately from your subject and predicate columns. If only 1% of the rows have the time, you pay only 1% of the cost. The null timestamps are stored extremely efficiently — both in disk size and in query efficiency. When you're querying for edges with a timestamp greater than X, we can skip over those nulls really efficiently.

When vectors and graphs disagree: “dog bit man” vs. “man bit dog”

Sunil: We only have a few more minutes. Sam, you want to…?

Sam: I have a couple of questions for Arun — I've been thinking through this for some time. One is about vectors. Vectors find things that look similar; they just say things are related. When they disagree, how does your LadybugDB trust model work?

Arun: Great question. Vectors group things by similarity, and sometimes similarity doesn't mean relevance. For example, “sugar” and “diabetes” may occur very closely in vector space because they're closely related — but when you store it in a knowledge graph, you can store edges that actually encode meaning: if you have diabetes, you want to be careful about sugar. Sometimes there are opposites — two concepts are related, but they're actually opposites of each other — and you use that in your retrieval. So when you do a graph-vector search, depending on whether you're making a $10 decision or a $100 million decision, you can use different modalities in the same system. Today people are forced to either go to the probabilistic system or the deterministic system — or read from the deterministic system and do all the reasoning probabilistically. This system gives you a little of both.

Sajjad: Sam, this is similar to the conversation people keep referring to — the sentences “dog bit man” and “man bit dog.” Semantically, they might appear in the same neighborhood, but with edges you can disambiguate the difference between the two.

Sunil: So you're saying the relationship between the words is the same in both sentences, but they mean completely different things?

Sajjad: Semantically, they might appear in the same region — with embeddings you might not be able to disambiguate between the two. But in a graph, with the edges and the direction, you can. Those two sentences might seem similar, so if you're searching for “dog bit man,” you might end up retrieving the opposite sentence, “man bit dog.” The edges help you disambiguate. What Arun is saying is that you need a combination of the two to do more evolved reasoning.

Arun: I'll take that example further. Give examples of a two-legged animal biting a four-legged animal. The question is, how do you translate “two-legged animal” to “man” and “four-legged animal” to “dog”? You have to use vector search, because it's not a full-text search. When you do the vector search, you get two sets of results — “man bites dog” and “dog bites man” — and then you apply more deterministic logic on top to find the right edge.

If the user is an LLM, not a human: what do you throw out?

Sunil: I think we've bitten off and eaten all the time we have for this podcast.

Sam: One last question. Suppose you were to design a database where the user is an LLM, not a human. What do you throw out — SQL, schemas, or transactions?

Arun: “Throw out” meaning you don't want the thing that's thrown out, right?

Sam: Exactly. The reason I'm asking is: what do you think is the fundamental problem with knowledge graphs today, and why did you start along this path? I want people to understand why you went there.

The real problem graph databases solve

Arun: Great question. The fundamental thing knowledge graphs solve is deep traversals and joins. With existing database technology, when you do a six-table join, there's an explosion of results — if you have 100 rows in every table, by the time you do your sixth join you're at 100 to the sixth power, which is a lot of rows. Why is this necessary? For traditional relational stuff — customers, orders, parts — the existing technology is good enough. The reason new technology is needed is if your goal is to take all the knowledge that's in the weights of an LLM, externalize it, and query it from a small language model. Existing database technology has many shortcomings — the most significant is the inability to do deep traversals and deep reasoning. That's why new technology has to be built, and agents, instead of humans, are a great use case for this technology.

From phones to data lakes: how far LadybugDB scales

Sunil: Okay, I think that's… sorry, Sajjad.

Sajjad: I just had one last question, Arun. With the model you have — similar to DuckDB, an embedded database that runs within a process — I heard some of the use cases for Kuzu were to run it within embedded devices, maybe a mobile phone. So the use cases can range from an embedded, lightweight device like a phone, to a single process in a server, to large-scale object stores with high-volume data, where you can have multiple instances of these nodes query that data, partitioning the data by partitioning the query across the dataset. So really, from embedded, to a single server, to a large-scale dataset — the ability to serve queries across all of these different use cases?

Arun: That's absolutely right. Ladybug has about 8 to 10 language bindings, and commits go through a CI that runs on an Android simulator and an iOS simulator. There are sample applications on how to embed it in a phone application, if desired. And then we're looking at how to scale out and distribute this in a data lake.

Wrap-up

Sunil: Again, I'm afraid that really is it. Thank you, Arun, for joining us — please hang on afterward, I'd like to chat with you for a second. Alright, so thank you all.