Part 25: Loss Function Explained: How Does a Neural Network Know It Is Wrong?
Imagine you are learning to bake a cake for the first time. You follow a recipe, but the cake turns out too sweet. The next time, you reduce the sugar. You taste it again and make further adjustments
Responses
Markdown supported · ⌘↵ to post
Yunus Emre Vurgun
Writing about AI agents, hardware, and practical software.
Nice clear walkthrough. A small worked example that helps MSE vs MAE click: take errors of 1, 1, 1 and 10 (in lakh). MAE = (1+1+1+10)/4 = 3.25, while MSE = (1+1+1+100)/4 = 25.75. The one big miss dominates MSE, which is why it punishes outliers hard and why MAE is more robust when your data has a few odd prices. A practical middle ground is Huber loss: it behaves like MSE for small errors and like MAE for large ones. And for cross-entropy, a fun check: a confident wrong answer (p=0.01 for the true class) costs about 4.6, versus about 0.69 for an unsure p=0.5.