A Single Neuron

Neural NetworksIntermediate~6 min

A Single Neuron — Weights, bias, and an activation function in one unit.

A single neuron takes inputs, multiplies each by a weight, adds a bias, and passes the result through an activation function. Master this one unit and you understand the atom of every deep network.

Low output
High output
Boundary (drag it)
Sample input

Sigmoid at z = 0.20 — drag left/right to move z (the bias follows)

Neuron controls

Data

Input to trace

Just off the boundary: z is close to 0, where the activation is steepest.

Or click any sample input on the plot; hover one to read its z and output.

Model

Activation

Weight w₁1.0

Weight w₂1.0

Bias b0.0

Drag the boundary line to aim the weights, or use the sliders — they stay in sync.

Playback

Step 0 / 3

Speed

Read input
Weighted sum
Add bias
Activate
↻

Step 0 of 3 — input (6.2, 4.4) centers to x = (0.40, -0.20)

Break it

Slope at this z0.248

The idea in plain words

A single neuron takes its inputs, multiplies each by a weight, adds a bias, and passes the result through an activation function. Master this one unit and you understand the atom of every deep network.

Drag the weights and the output surface tilts and shifts. Push the weights very high and the activation saturates flat — its gradient goes to zero, so the neuron stops learning. That’s a preview of the vanishing-gradient problem.

Now, the math

The neuron’s output is an activation of a weighted sum plus bias:

a = f\!\left(\textstyle\sum_i w_i x_i + b\right)

$w_i$: the weight on input i — its importance and sign.
$b$: the bias — shifts the threshold.
$f$: the activation — the nonlinearity.

▸ Show the derivation

The weighted sum defines a plane; the activation bends it into an output surface. Large weights make that surface steep, pushing most inputs into the flat, saturated tails where the derivative — and thus the learning signal — is nearly zero.

Trace it by hand

One neuron with two inputs x = (2, 1), weights w = (0.4, -0.6), bias b = 0.5, and a sigmoid activation f. Activation values are rounded to 4 decimal places; they come from the repo's activation core.

Compute the weighted sum
$\textstyle\sum_i w_i x_i = 0.4(2) + (-0.6)(1) = 0.8 - 0.6 = 0.2$
The positive weight votes for firing, the negative weight votes against; the votes nearly cancel.
Add the bias to get the pre-activation
$z = \textstyle\sum_i w_i x_i + b = 0.2 + 0.5 = 0.7$
The bias shifts the threshold — here it pushes an almost-neutral sum firmly onto the positive side.
Pass z through the sigmoid activation
$a = \sigma(0.7) = \frac{1}{1 + e^{-0.7}} \approx 0.6682$
The sigmoid squashes any real number into the interval from 0 to 1.
Read the output as a probability
$a \approx 0.6682 \;\Rightarrow\; \text{about a } 67\% \text{ chance of class } 1$
Leaning positive but far from certain — z would need to be much larger for the output to saturate near 1.

What just happened: Weighted sum 0.2, plus bias 0.5, through the sigmoid gives 0.6682: every deep network is just this three-beat pattern — multiply, add, squash — repeated millions of times.

Now Break It

Try this: Saturating the activation (huge weights) flattens the gradient to zero — the neuron stops learning.

Control: Weight magnitude slider (set very high)

What happens: Saturated! Huge weights push the activation flat — the gradient vanishes and learning stalls.

Where a single neuron is used

A single artificial neuron bundles three ideas into one compact unit: weights that scale each input by its importance, a bias that shifts the decision threshold, and an activation function that transforms the result. This tiny building block is remarkably capable on its own. A single sigmoid neuron is exactly logistic regression, a workhorse used for credit scoring, medical risk prediction, and click-through estimation across the web. Because the neuron is differentiable, it can be trained with gradient descent, and because it is cheap to evaluate, millions of them can be composed into the layers that power image recognition and language models. Grasping one neuron in full detail is the shortest path to understanding an entire network.

A frequent misconception is that a neuron simply passes its weighted sum forward; in reality the activation function is what gives the neuron its character, and without it the unit is purely linear. Another pitfall is confusing weights and bias: weights control the orientation and steepness of the decision boundary, while the bias shifts it away from the origin, and dropping the bias forces every boundary through zero, often crippling the fit. Beginners also assume a single neuron can model complex relationships, but on its own it can only represent a linear combination passed through one nonlinearity, which is why real problems demand many neurons arranged in layers.

Frequently asked questions

What are the parts of a single neuron?

A neuron has weights, a bias, and an activation function. It multiplies each input by its weight, sums those products, adds the bias, and then passes the total through the activation function to produce its output. The weights and bias are the learnable parameters adjusted during training.

What does the bias term do?

The bias lets a neuron shift its decision boundary away from the origin, so the output does not have to be zero when all inputs are zero. Without a bias, every boundary would be forced through the origin, sharply limiting what the neuron can represent. It works like the intercept term in a linear equation.

What is the difference between weights and bias?

Weights determine how strongly each input influences the output and set the orientation and steepness of the boundary. The bias is a single extra number that shifts the whole boundary up or down independent of the inputs. Both are learned from data, but they play distinct geometric roles.

Can a single neuron learn anything useful?

Yes, within limits. A single sigmoid neuron is equivalent to logistic regression and can classify data that is linearly separable, which covers many practical tasks. But it cannot capture nonlinear patterns on its own, so complex problems require many neurons combined into layers.

How is a neuron trained?

Training adjusts the weights and bias to reduce the difference between the neuron's predictions and the true targets. This is usually done with gradient descent, which computes how each parameter affects the error and nudges it in the direction that lowers the loss. Repeating this over many examples gradually improves the neuron.

← Back to all visualizations Continue on the Learning Path →

Written & reviewed by the ML Visualization team · Last updated August 6, 2026.