Tag

#moe

16 posts0 followers

Explore Hashnode

Alternatives

KhaskySep 12 · 5 min read

Edge0 Streams MoE Experts off SSD to Fit 35B in 3 GB

One question took me through Edge0: if the 35B model only needs 2.9 GB of RAM, what does the machine actually need? The answer is a disk budget, a cache budget and a context budget, and none of them i

khasky.hashnode.dev

Botao DengAug 27 · 19 min read

From Mixtral to Kimi K3: How Mixture-of-Experts Models Evolved

In this article, we'll discuss how Mixture-of-Experts models grew from a handful of experts to nearly 900 per layer, and the compression and stability mechanisms that keep such a sparse design trainab

freecodecamp.org

Neeloppher SyedJul 21 · 8 min read

MoE Cost Analyzer: Benchmark Dense vs Mixture-of-Experts LLMs on Your Own Prompts with Real API Calls

Every team running LLMs in production eventually asks the same question: should we switch to a Mixture-of-Experts model? The MoE architecture activates fewer parameters per token, which means cheaper

1 neeloppher.hashnode.dev

Learn with HJJul 11 · 6 min read

Mixture of Experts: Why Some Models Use Specialist Sub-Brains

🚀 Mixture of Experts: Why Some Models Use Specialist Sub-Brains 👋 Welcome to Day 67 of 90 Days of AI. 🎯 Today we are tackling Mixture of Experts: Why Some Models Use Specialist Sub-Brains. The mis

hardeepjethwani.hashnode.dev

Devarsh PatelMay 29 · 13 min read

I Ran a monster GPT-OSS-120B MoE AI Model on my MacBook — Local LLM(No Cloud GPU) with llama.cpp

💡 Running a large language model (LLM) locally gives you absolute privacy, zero ongoing subscription costs, and the ability to work entirely offline. It is ideal if you have strict data security need

devarshpatel.hashnode.dev

Jangwook KimMay 11 · 10 min read

ZAYA1-8B: Zyphra's Efficient MoE Reasoning Model Guide

The scaling-is-everything story has a new challenger. On May 6, 2026, Zyphra released ZAYA1-8B — an open-weight Mixture-of-Experts reasoning model with 8.4 billion total parameters and fewer than 800 million active per token. On AIME 2025, a benchmar...

effloow.hashnode.dev

Jangwook KimMay 5 · 6 min read

Mistral Large 3: The 675B Open-Weight MoE Model Developer Guide

Mistral Large 3 launched in December 2025 as Mistral's flagship open-weight model. Six months later it remains the largest model Mistral has publicly released under a permissive license. This guide covers the architecture, benchmarks, pricing, and pr...

effloow.hashnode.dev

Jangwook KimApr 28 · 11 min read

vLLM 0.8: Native Llama 4 MoE Routing Explained

Mixture-of-Experts models have dominated the open-weight frontier in 2026. Llama 4 Scout (17B-16E), Llama 4 Maverick (17B-128E), DeepSeek V4-Pro (1.6T-49B active), and Qwen3.6-Plus all use sparse expert routing to scale parameters without proportiona...

effloow.hashnode.dev

Jangwook KimApr 25 · 11 min read

DeepSeek V4-Pro and V4-Flash: Migration Guide and API Setup

DeepSeek dropped two new models on April 24, 2026: V4-Pro, a 1.6-trillion-parameter MoE flagship, and V4-Flash, a 284-billion-parameter workhorse optimized for throughput. Both support a one-million-token context window, dual Thinking/Non-Thinking mo...

effloow.hashnode.dev

Jangwook KimApr 22 · 11 min read

Qwen3.6-Plus: 1M Token Context and Claude-Level Performance

## Why This Matters Every few months, a model drops that forces you to recalibrate your mental model of what "frontier-level" performance costs. In April 2026, that model is Qwen3.6-Plus from Alibaba. The headline numbers: a 1-million-token context...

effloow.hashnode.dev

Search Hashnode

Explore Hashnode

Trending tags this week

Edge0 Streams MoE Experts off SSD to Fit 35B in 3 GB

From Mixtral to Kimi K3: How Mixture-of-Experts Models Evolved

MoE Cost Analyzer: Benchmark Dense vs Mixture-of-Experts LLMs on Your Own Prompts with Real API Calls

Mixture of Experts: Why Some Models Use Specialist Sub-Brains

I Ran a monster GPT-OSS-120B MoE AI Model on my MacBook — Local LLM(No Cloud GPU) with llama.cpp

ZAYA1-8B: Zyphra's Efficient MoE Reasoning Model Guide

Mistral Large 3: The 675B Open-Weight MoE Model Developer Guide

vLLM 0.8: Native Llama 4 MoE Routing Explained

DeepSeek V4-Pro and V4-Flash: Migration Guide and API Setup

Qwen3.6-Plus: 1M Token Context and Claude-Level Performance