For variable request costs, I would distinguish an empty bucket from a request that costs more than the bucket can ever hold. Waiting and retrying can fix the first case but not the second, so the client needs a different explanation
Absolutely. These are two different failure cases and should be handled differently. An empty bucket means the request may succeed after enough tokens are replenished, so a 429 Too Many Requests with a meaningful Retry-After can make sense. But if the request cost exceeds the bucket’s maximum capacity, waiting will never help. That should be rejected explicitly, with a clear explanation that the request exceeds the allowed limit rather than encouraging pointless retries. Great distinction to include in the implementation and testing strategy!