How to Find the Agent Actions a Prompt Injection Can ControlTL;DR: A model treats every word in its context as a possible instruction, including text a tool fetched. Any decision your agent makes after reading that text belongs to whoever wrote it. Engineer's 3d ago · 9 min readtheaicommit.hashnode.dev
How to Correct an LLM Judge's Pass Rate With 100 Human LabelsTL;DR: The pass rate a grading model reports is part your product and part the grader's own mistakes. Measure those mistakes on a small set a person has checked, and you can subtract them back out. En22KA5d ago · 10 min readtheaicommit.hashnode.dev
Why Forced JSON Output Is Always Valid and Sometimes WrongEngineer's view. This is a required dropdown with no "Not applicable" option. Validation always passes, so the data looks clean. But users who had no right answer picked the first item, and your repor22KGOct 24 · 9 min readtheaicommit.hashnode.dev
Why Split Prefill and Decode GPUs Lose Time in the Hand-Off, Not the NetworkEngineer's view. This is a Node.js server doing a synchronous parse on every upload. Each parse is quick, but it blocks the one loop every connection shares, so every response slows. With a fixed connOct 23 · 10 min readtheaicommit.hashnode.dev
Designing a per-user AI cost meter: idempotency, cache pricing and aliasesAI apps have a margin problem that normal SaaS metrics hide: price per user is flat, cost per user is variable. I built FiscaLLM (MIT) to join the two. Most of the work wasn't the dashboard. It was maOct 23 · 3 min readtheaicommit.hashnode.dev
