Neural networks are just very long functions. This post tries to give you a good mental map of how they work. I trained a small neural network on MNIST, a famous dataset of 70,000 handwritten digits, and put it into this page. If you draw a digit in the black box below this paragraph, the model will guess what number you drew.
The model doesn’t have eyes and thus cant see a picture. Your drawing gets shrunk down to a grid of 28 by 28 pixels, and each pixel is converted into number describing how bright it is. 0 meaning completely black, 1 meaning completely white, That is the actual input the model takes in input. Your drawning is now 784 numbers (28^2).
The grid below shows your drawing after that step, if you hover over it you can read the actual numbers. Press the button underneath to flatten the grid into one long row, because that is what the model really receives. This list of numbers.
Once the drawing is a list of numbers, the prediction is made with math. The model multiplies each of the 784 numbers by a “weight”, adds everything up, and repeats that ten times, once for each digit from 0 to 9. The biggest total wins. Written as math it’s f(x) = softmax(W·x + b) You could do every one of them on paper and get the exact same answer the model gets.
So the last question you may have is: what is a “weight”? A weight is just a number the model stores. There is one for every pixel and every digit, 7,840 in total. Below I drew the weights of my model as pictures, one image per digit. An orange pixel means drawing there adds to that digit’s score, and a blue pixel means drawing there subtracts from it. If you look at the 0 image you can make out the shape of a 0.
I didn’t set these weights, and neither did anyone else. They were generated by an algorithm called gradient descent, which is what the next post covers.