Muhammad Ahmad

Good point, the timeout only stops the waiting, not the work. For write tools, an AbortSignal plus idempotency keys would stop a retry doing the same thing twice.

Good idea, that line does get lost in a long session. A count across sessions would also show which errors the agent keeps failing on.

Agreed, the Stop hook only proves what the checks cover. Hand-written rules sit outside that, so keeping them away from anything generated is the best answer I have too.

Good catch, the wrapper only logs on success, so a dropped stream disappears from the numbers. A finally block with the last known usage and a status field is the right fix.

Cost per completed task is the better number, retries hide in the per-call view. I'd re-run the eval on a schedule. Input mix drifts quietly, and a route that was safe to downgrade can stop being safe.