The Loss Landscape

Gradient descent is a ball rolling downhill. Its famous weakness: it stops in the first valley it finds. Shake it out.

Learning rate Momentum Shake power Reveal global minimum

Loss–

Best–

Global min???

Shakes0

Press Drop ball to start.

You found the global minimum