MMatthew1d ago · 6 min readApply Backpressure to AI API WorkloadsDisclosure: I work on Modelflare, an OpenAI-compatible AI API gateway. The canonical version of this article lives on modelflare.dev. Apply Backpressure to AI API Workloads: a production guide with an2Amatthewmodelflare.hashnode.dev
AAumCreate1d ago · 8 min readForm submissions into a CRM without losing the ones that failThe thank-you message renders, the form clears, and the lead is gone. Somewhere between the submit handler and the CRM record there was a timeout, a 200 response carrying an error inside the body, or aumcreate.hashnode.dev
NnRouter.ai1d ago · 4 min readLowest Recent p95 Wins: Latency Routing Without Retry CodePin an application to one model name and you inherit that provider's worst day. This article covers the gateway configuration that replaces per-service retry code: a router alias with the Latency stra1Enrouter.hashnode.dev
VVirginia Mwega2d ago · 5 min readRetries Are a Latency Budget, Not a BooleanRetries Are a Latency Budget, Not a Boolean A retry can make a transient failure invisible to the user. It can also turn a single slow request into a much longer wait, multiply network traffic, and ma23KEAvirginiamwegahashnodedev.hashnode.dev
DDamon Long2d ago · 16 min readFrom Benchmark Scores to Reliable Personal AILately, one question has become harder for me to ignore when using large language models: why does a model that feels impressive at launch not always feel equally reliable after I have used it for a wdamonlong.hashnode.dev
NnRouter.ai3d ago · 5 min readYour Retry Logic Is Retrying a Budget: Branch on the Code, Not the StatusHere is a failure mode that hides in plain sight: a key hits its budget ceiling at 2 a.m., the gateway refuses the next request with a 429 — and your client's exponential backoff does exactly what it nrouter.hashnode.dev
AAbir Eltaief5d ago · 57 min readArchitecting for Resilience: Battle-Tested Reliability Patterns Across AWS, Azure, and GCP (Part 4/5)Series Foreword: Demystifying Complex Cloud Systems Through Design Patterns As an active Multi-Cloud Architect across AWS, Microsoft Azure, and GCP—having recently earned my Google Cloud Certified Prscalability-patterns-modern-distributed-systems.hashnode.dev
PPowerduck6d ago · 4 min readThe agent retried the whole task after one failed tool call and duplicated the side effectsA recurring job fetched an unpaid order, charged the card through the gateway, then sent a receipt email. On one run the payment call timed out at the network edge. The wrapper caught the exception, w1blogs.powerduck.com
NnRouter.aiOct 23 · 4 min readA 429 Is Not Always a Throttle: Budget Blocks and Retry Logic That Knows the DifferenceYour client code treats every 429 the same way: back off, retry, move on. Against a rate limit that is correct. Against an LLM gateway's budget enforcement it is a bug — one that adds latency and loadnrouter.hashnode.dev
TTirth GajjarOct 21 · 14 min readWhy AI agents report unfinished work as doneAn action-taking agent can receive a 200 OK while the customer’s task remains incomplete. The API may have accepted the request while a queue still rejects it, a downstream system fails, a browser sub12KAbigcircle.hashnode.dev