Forward Propagation
Forward Propagation — Push inputs through the layers to compute a prediction.
Forward propagation is how a network makes a prediction: feed the inputs into the first layer, pass the outputs to the next, and repeat until the final layer produces an answer. Just matrix multiplies and activations, layer by layer.
Drag the input point (or use the x₁ / x₂ sliders) and watch the activations ripple forward.
Step the activation wavefront layer by layer. Poor weight initialization makes activations explode toward the saturation limits or vanish toward zero as they propagate forward.
Step the activation wavefront layer by layer. Poor weight initialization makes activations explode toward the saturation limits or vanish toward zero as they propagate forward.
The idea in plain words
Forward propagation is how a network makes a prediction: feed the inputs into the first layer, pass its outputs to the next, and repeat until the final layer produces an answer. Just matrix multiplies and activations, layer by layer.
Step the wavefront and watch each neuron light up with its value. Poor weight initialization makes those values explode toward the saturation limits or vanish toward zero as they propagate — which is why initialization schemes matter.
Now, the math
Each layer transforms the previous layer’s activations:
- the weight matrix of layer l.
- activations arriving from the previous layer.
▸ Show the derivation
If weights are too large, repeated multiplication amplifies the activations layer after layer until they saturate; too small, and they decay toward zero. Initialization schemes (Xavier, He) scale the weights by the layer width so activation variance stays roughly constant with depth.
Now Break It
Try this: Poor weight initialization makes activations explode or vanish as they propagate forward.
Control: Weight init scale slider (set very high or low)
Last updated .