Temporal Context Graphs and Your AI App
The database you didn't know you needed.
One of the most transformative moments for me as a developer was coming across immutable data structures in Clojure and learning that you can reshape problems in a way that helps you make significantly less mistakes as an engineer.
In most programming languages data structures are mutable just like they are in computer hardware. This gives you the most flexibility but it also means that it’s really complicated to reason about changes.
When data structures become immutable, it's more difficult to build things but once you do they are more likely to be correct since changes are minimized.
Datomic turned this idea into a database that stores facts/events and then computes over them. Nothing ever gets removed since it doesn't need to. When you add something, you create an add event and when you remove something you add a remove event. Your interpretation of the data may be a mutable data structure but the underlying data is immutable (which is comforting since it prevents the worst kind of mistakes – the irreversible ones).
Since LLMs can reason about facts in more complex ways, a generalization of this type of database is now possible.
I've been building simple versions of this in my AI repositories. For example I may produce a piece of research, store it in a markdown file and then ask future queries to update the research. I then create analyses files that are updated in line with the research content.
At some point you need a real database. It could be to create memories/context for users, to access the data from different surfaces or just to query it more efficiently.
So I started looking for a more scalable solution and found Graphiti. This is exactly the space they seem to be exploring in.
It’s a database that will use inference to not just store raw data but what you can learn from it.
Imagine you read a 300-page book and you submit each page as an “episode” to the database. From there you could ask questions about the book using queries and Zep/Graphiti will prepare a context block that only features the most relevant pages to answer the question.
But that’s not all. At the point in time where you submitted the pages to the database, Zep/Graphiti started synthesizing higher order forms of context. It reasons about entities, stores facts, makes observations, all based on the facts you provide.
In a regular database, all you get would be a 300-entry table and to find answers you would likely have to loop through every single page and use each of the pages in a prompt and then synthesize.
With Zep/Graphiti, it may already have the answer you are looking for stored in the database itself which would give you an incredibly low-latency response.
While motivated by the common case of user memory and context, I think this has many applications beyond that. Different LLM apps will want to maintain their own interpretation of the world and that could include views on entities different from its users.
The best thing about temporal context graphs is just how much room there is to architect them in different ways since you need to use inference at the point of ingestion.
I'm excited to try Zep out soon and I believe they are one of the most overlooked pieces of infrastructure for AI applications right now (and not just for teams that want user-specific memory).