Journal
Does Graphify Actually Cut Token Usage?
For a few weeks my codebase carried a passenger. It was called graphify: a tool that reads an entire repository and builds a knowledge graph of it—every function, every file, every call between them—so that an AI agent could ask "how does authorization work here?" and get an answer without opening a single file. I had wired it in aggressively. The graph was committed to the repo, all 15 megabytes of it, and a hook nagged every agent to consult it before it was allowed to read source code.
The promise was simple and seductive: reading code is expensive, and the biggest expense when you work with AI agents is tokens—the units of text they read and write, which is what you ultimately pay for. If a compact graph could answer questions that would otherwise take dozens of file reads, it should save tokens. A lot of them.
I had never actually checked whether it did. So I checked.
The test
I wanted a fair fight, on exactly the kind of question graphify is supposed to win: something that crosses many parts of the system at once. I picked a real one from my hotel-management codebase:
Trace how an incoming authenticated request is authorized—from the login cookie all the way to the specific permission check—and name the exact functions and the files they live in, in the order they run.
That question touches four different modules. To answer it you have to understand how the pieces connect, not just where one function lives. Perfect terrain for a graph.
Then I set two AI agents loose on it, in isolation, with identical instructions, each allowed only one method:
- Agent A could use only graphify. No reading files, no searching.
- Agent B could use only the ordinary tools—searching the text and reading the files, the way anyone would if they'd never heard of graphify.
Afterward I checked both answers against the actual source code, line by line, to see who was right and how much each had cost.
What came back
Here are the headline numbers.
| Graphify | Search + Read | |
|---|---|---|
| Tokens used | 35,215 | 37,068 |
| Confidence in its own answer | Medium | High |
| Execution order | Guessed | Verified against the code |
| Completeness | Missed the two functions that do the work | Full chain |
On the face of it, graphify won: it used about 1,850 fewer tokens, roughly 5% cheaper. If I'd stopped reading at the first row of that table, I would have kept the tool.
But the 5% is a mirage.
The cheaper answer was the wrong answer
Both agents correctly named the three "gatekeeper" functions that sit at the front door—the middleware that checks that you're allowed in. So far, a tie.
The difference is what happened next. The search-and-read agent kept going and found the three functions that actually perform the authorization: the one that validates your login cookie, the one that resolves what permissions you have, and the one that runs the final "can this user do this?" check. All three verified correct against the source.
Graphify never surfaced two of those three. It produced the skeleton—the outer shell of gatekeepers—but not the logic inside, the part that actually decides whether you're allowed to do the thing. And it knew it was unsure: it flagged its own ordering as a guess.
That reframes the whole comparison. The reason graphify was cheaper is that it quit early with a vaguer answer. If you actually needed to do something with the result—fix a bug in the auth path, say—you'd still have to open the files to be sure, because a medium-confidence, incomplete answer isn't safe to act on. So the true cost of the graphify route is its 35K tokens plus the file reads you have to do anyway. That's more than the 37K the other agent spent to be certain the first time. The saving only exists in the world where you never needed to be right.
It got worse the closer I looked
I also ran graphify's own commands directly, to see each mode on its own terms. The results were unkind.
- Its one genuinely good mode was looking up a symbol by its exact name. It returned the right location and the real connections. But that's just a fancier version of searching for a word—the ordinary tools do it just as well.
- The mode the hook forced everyone to use—asking a question in plain English—was the worst of the lot. Ask it about "permission resolution" and it returned two dozen results about pricing, because it latched onto the word "EffectivePrice" while I'd asked about "permission." Ask about the login flow and it pointed me at an unrelated scheduling feature and a documentation file. Mostly noise.
So the tool's most reliable mode was redundant, and its mandatory mode was its weakest. That's close to the worst possible arrangement.
And it was already out of date
There's one more problem, and it's structural. A graph is a photograph of the code at one instant. Mine was built at 5:13 one afternoon. Six hours later I committed a change to—of all things—the exact permission-resolution function the test was about. The graph never knew. It was stale the same day it was born, and every commit afterward widened the gap.
Keeping it fresh isn't free either. Rebuilding the graph from scratch costs roughly 572,000 tokens—many times the entire test I just ran—for one repository, once. You'd have to run broad, graph-shaped queries constantly for that to ever pay back, and the daily reality of coding is small, targeted edits, not sweeping tours of the architecture.
Why it felt useless even before I measured it
The numbers confirmed a hunch I'd had for a while.
The hook fired on everything. It demanded I run graphify before every file read—including entire sessions of configuration and tooling work that had nothing to do with understanding the codebase. A tool that interrupts you 100% of the time to help you a fraction of the time reads as pure friction, however clever it is underneath.
And there's a quieter reason. Graphify's real gift is orienting a stranger in a large, unfamiliar codebase—someone parachuting in who needs a map. But this is my own repository. I built it. I don't need a map of a house I laid the foundations for.
Where it would earn its keep
To be fair to the tool: this isn't "never useful." It's "not useful here." It would genuinely pay off when several things are true at once—a large codebase you don't know, broad architectural questions about how things flow across it, asked often enough to amortize that half-million-token build, and used to get your bearings rather than to produce exact, ready-to-edit detail. On a repo you know, doing mostly small edits, none of that holds.
What I did about it
On the same day, I pulled the machinery out:
- Removed the hooks that forced graphify before every read.
- Stopped committing the graph. That 15 MB file is gone from the repository, which also quietly retired a whole separate headache about how to ship such a large file to cloud sessions.
- Rewrote the docs to describe graphify as optional and on-demand, with plain searching and reading as the default.
The tool is still installed for the rare moment I want to look up a single symbol. It's just no longer committed, no longer forced, and no longer standing between me and my own code.
The lesson I'm taking isn't "knowledge graphs are bad." It's that a saving you never measured is just a story you told yourself. The moment I put a number on it, the number said the quiet part out loud: the cheaper answer was cheaper because it was worse.
