A Single Neuron

Neural NetworksIntermediate~6 min

A Single Neuron — Weights, bias, and an activation function in one unit.

A single neuron takes inputs, multiplies each by a weight, adds a bias, and passes the result through an activation function. Master this one unit and you understand the atom of every deep network.

Activation

Weight w₁1.0

Weight w₂1.0

Bias b0.0

One neuron weights its inputs, adds a bias, and passes the sum through an activation. Drag the boundary line to aim the weights, or use the sliders — they stay in sync. Crank the weights very high and the activation saturates flat: its gradient vanishes and the neuron stops learning.

Activation

Weight w₁1.0

Weight w₂1.0

Bias b0.0

The idea in plain words

A single neuron takes its inputs, multiplies each by a weight, adds a bias, and passes the result through an activation function. Master this one unit and you understand the atom of every deep network.

Drag the weights and the output surface tilts and shifts. Push the weights very high and the activation saturates flat — its gradient goes to zero, so the neuron stops learning. That’s a preview of the vanishing-gradient problem.

Now, the math

The neuron’s output is an activation of a weighted sum plus bias:

a = f\!\left(\textstyle\sum_i w_i x_i + b\right)

$w_i$: the weight on input i — its importance and sign.
$b$: the bias — shifts the threshold.
$f$: the activation — the nonlinearity.

▸ Show the derivation

The weighted sum defines a plane; the activation bends it into an output surface. Large weights make that surface steep, pushing most inputs into the flat, saturated tails where the derivative — and thus the learning signal — is nearly zero.

Now Break It

Try this: Saturating the activation (huge weights) flattens the gradient to zero — the neuron stops learning.

Control: Weight magnitude slider (set very high)

← Back to all visualizations Continue on the Learning Path →

Last updated July 3, 2026.