Amaresh Pelleti

Good catches, both worth flagging. You're right that Ollama's context default scales with VRAM (4,096 under ~24GiB, which is exactly where most single-developer/laptop setups land — the use case this article points people toward for Ollama), and it truncates silently rather than erroring, so a benchmark can look clean on short prompts and quietly degrade on longer ones. OLLAMA_CONTEXT_LENGTH is the right lever for a server-wide default; num_ctx overrides it per-request or in the Modelfile. The warm-up point is well taken too — cold model-load time folded into a single request is a classic way to make steady-state throughput look worse than it is. Might work both into a future revision of this piece.

That split holds up well in practice, and the switch between them is close to free since both speak an OpenAI-compatible API — swapping the base URL doesn't mean rewriting your client. One gotcha if you're doing quality comparisons between local and cloud: Ollama's default model tags are Q4_K_M, so "the cloud model is smarter" is sometimes just "I compared a 4-bit quant to an 8-bit one." Worth pulling a q8 tag before judging.

Model tiering is the one that actually moves the needle — routing routine edits to a cheaper model instead of always hitting the frontier one is where most of the savings are. One thing worth checking before reaching for a separate gateway: GitHub's own Copilot billing dashboard already shows projected usage and lets you set a hard budget cap on overage, so you get most of the "track everything" benefit built in.

Good add — the hidden-defaults problem is worth naming specifically for Ollama: it sizes context off available VRAM unless you set num_ctx yourself, so the "default" run of the same model can silently get less context on one machine than another, independent of which tool it's in. On memory pressure, that matches what I've seen too — Ollama's llama.cpp path offloads layers to CPU and just gets slower, while LM Studio with strict GPU-only offload set will refuse instead of degrading. Slow-but-working vs. clean failure is a good frame — might be worth its own post.