Ziwei Yao

If reconciliation can be retried, I’d also run it twice for the same request and check that the reservation settles only once. That would cover duplicate settlement as well as the retry race.

A latch in the test could make the pause reproducible instead of depending on sleep timing. That would be a nice companion to the revised example.

Thanks! The accounting edge case I'd keep explicit is 'client disconnected, provider outcome unknown'. Releasing that reservation immediately could let a retry spend the same budget twice. I'd reconcile it from the provider's final usage or send the unknown outcome for review, rather than treating a browser disconnect as proof that no work was billed.

Thanks for checking that. I'd have the regression test assert that the stale holder's write is rejected after the newer holder has written, not just that stock stays non-negative. Those are separate guarantees, and the stock constraint could otherwise make the test pass while fencing is still broken.

Should lease acquisition and fencing-token allocation be atomic? With SET NX followed by a separate INCR, A could acquire, pause before INCR, and lose its lease. B acquires, gets token 1 and writes. A resumes, gets token 2, and its stale write passes fence_token < :token. The stock check still prevents overselling, but does this break the stale-holder guarantee? I’d test a pause between those Redis commands.

One extra layer I’d add for long-running LLM requests is an in-flight concurrency limit. A token bucket caps arrivals, but if calls become much slower, the number running at once can still grow while the arrival rate stays within budget. For usage that isn’t known up front, reserving an upper bound and reconciling with actual usage also needs its own accounting. I’d test a slowdown and a client disconnect separately: the client going away doesn’t necessarily mean the provider stopped doing work.