Discussion on "Day 03: Your repo does not fit in the context window. Now what?"

Harshavardhan Katkam · 2026-10-14T07:21:10.948Z

Handing cartographer files one at a time had been working, so I did the obvious next thing and wrote the four lines that walk a repository and concatenate every source file into one string. Large cont

Two samples and a subtraction to cancel the chat template overhead is such a cheap fix, and I had been dividing by a single sample without ever thinking about it. The line about the transcript re-sending itself on every call matches what hurt my agent costs, long before the window itself filled up. Do you re-measure the ratio only when you switch models, or on every run?

Good practical framing. One thing worth adding: in tool-heavy sessions the dominant token cost often isn't the repo at all — it's tool outputs (test logs, command results, large diffs) that pile up in the transcript. A meter that only tracks repo size can understate real pressure, so tracking both sides of the ledger makes the budget more honest.

Agreed, and it's a point the series runs into head-on. One clarification on Day 03: the repo size is only the up-front estimate. The meter itself records the usage the server reports on every request, so whatever is in the transcript gets counted, tool outputs included. Neither side of the ledger is guessed.

You're right about where the pressure moves, though. Once the agent can call tools (Day 06), file reads pile up in the transcript and soon outweigh everything else. Day 08 is about exactly that: tools that return a bounded slice of a file instead of the whole thing. A later memory day handles what to drop from a long transcript without breaking it.

I don’t have usage-cost data to cite for orke, but for multi-agent workflows I’d split the meter by task and handoff as well as input/output. A whole-run total can hide whether cost came from generating once or resending shared context across steps. I’d validate an estimate against a small representative run and record the model/version so it doesn’t silently drift. Does your meter preserve per-request usage for that breakdown? I’m with BOMTOBE, makers of orke.

Good question, and an honest answer: not yet. The Day 03 meter keeps running totals only (calls, input tokens, output tokens, cost), because the point that day was the first number, not the breakdown. You're right that a total hides where the cost came from. Here, almost all of it is the transcript being resent on every call.

The breakdown comes later in the series. A LangGraph day counts calls per node and per run. The tracing day records each model call as its own span, with its usage, so you can see which step resent what. On drift: every measurement is recorded with the model and the commit it was taken at, for exactly that reason.

That progression makes sense: establish the baseline, then attribute calls by node/run and trace. For the tracing day, do spans keep a parent task/run ID across handoffs? I’m curious whether that makes repeated context visible on a child step without blending it into that step’s own prompt.

Really useful breakdown, especially the point that measuring characters or using the wrong tokenizer can give a misleading picture of an AI agent’s actual cost. I like how you turn an easily overlooked technical detail into a practical lesson about budgeting, context limits, and smarter system design. At Oglas AI, we believe building effective AI solutions means thinking beyond the model itself and paying attention to efficiency, reliability, and real world costs. Looking forward to the next part of this series!

Thanks! The measured number surprised me too: on the free model I'm using, code came out at about 2.58 characters per token, well below the usual rule of thumb. That's why the ratio gets calibrated, not assumed. Next up, Day 04: making the model return structured output instead of prose I have to parse.

Search Hashnode

Day 03: Your repo does not fit in the context window. Now what?

Responses8