KAI Scheduler Deep Dive: How NVIDIA Schedules GPUs on KubernetesWhy GPU clusters need a different scheduler The default kube-scheduler places one pod at a time, on one node, using a filter-and-score loop. That model is excellent for stateless microservices and wro23AAOct 22 · 35 min readblog.shubhamtatvamasi.com
NVIDIA Dynamo vs KServe: Which Layer Should Own Your LLM Inference?In my previous article I split a production inference platform into layers: LiteLLM for tenants and budgets, Envoy AI Gateway for model-aware routing, KServe LLMInferenceService for lifecycle, llm-d fOct 19 · 13 min readblog.shubhamtatvamasi.com
Running LLMs on Kubernetes in Production: KServe, vLLM, llm-d, Envoy AI Gateway, and LiteLLMSelf-hosting an LLM on Kubernetes is easy to demonstrate. Operating a fleet of models under real traffic is a different problem. A production inference platform must answer several questions at once: 2Oct 11 · 26 min readblog.shubhamtatvamasi.com
Build for 10,000 YearsMost people working in tech today don’t think about legacy. They focus on finishing the sprint, shipping the next release, or just getting through the week. Their contribution often stops at what’s assigned, and rarely do they pause to understand the...Jun 14, 2025 · 2 min readblog.shubhamtatvamasi.com
