Building a Modern LLM from Scratch: RMSNorm, RoPE, GQA and SwiGLUThis is the fourth part of a series on how language models work, built from the ground up. Part 1 built a neural network as an office building and trained it. Part 2 put text through the building and Oct 13 · 43 min readmaheshmoholkar.hashnode.dev
The Math Underneath Neural Networks: Tensors, Matrices and PyTorchThis is the third part of a series on how language models work, built from the ground up. Part 1 built a neural network as an office building and trained it with four steps on repeat. Part 2 put text Sep 28 · 44 min readmaheshmoholkar.hashnode.dev
Text to Attention From ScratchThis is the second part of a series on how language models work, built from the ground up. The first part covered what a neural network is made of and how it learns: an office building of workers, manSep 12 · 70 min readmaheshmoholkar.hashnode.dev
Neural Networks from ScratchI spent a long time nodding along to neural network explainers without actually understanding them. I could repeat the words — weights, backpropagation, activation function — but I couldn't have told Sep 10 · 53 min readmaheshmoholkar.hashnode.dev
Observability Driven Development with Sprkl🙂 What is Observability Driven Development ? Development is kind of like an iterative process. You write some code, you make some changes and then you try to execute it. After execution you try to analyze the result and sometimes you find that it i...Oct 18, 2022 · 2 min readmaheshmoholkar.hashnode.dev
