Skip to content
ML Visualization

Logistic Regression

ClassificationBeginner~7 min

Logistic Regression — Predict class probabilities with an S-shaped curve.

Despite the name, logistic regression is for classification. It squashes a linear score through an S-shaped sigmoid to output a probability between 0 and 1, then draws a decision boundary.

  • Class 0
  • Class 1
  • Boundary (p=0.5)
Sigmoid: score → probability (hover a point on the map)
Log-loss vs iteration
0.6930Max log-loss on axis: 0.6931

Logistic controls

Data
Dataset
26
1.0×
Click action
Model
0.60
900
Training accuracy50%
‖w‖0.00
Playback
Step 0 / 40
Speed
  1. Weights at zero
  2. Descending
  3. Settled

Step 0 of 40 — after 0 epochs at learning rate 0.60 — log-loss 0.693, ‖w‖ 0.00, training accuracy 50%

Break it

Perfectly separated classes have no finite best answer: every epoch makes ‖w‖ larger and the sigmoid steeper, chasing a loss that never reaches zero.

The idea in plain words

Despite the name, logistic regression classifies. It takes the same linear score as linear regression and squashes it through an S-shaped sigmoid into a probability between 0 and 1. The boundary is where that probability crosses 0.5.

Drag a point and the whole probability field re-forms. If the classes are perfectly separable the fit keeps pushing the weights larger and larger to make the sigmoid ever steeper — a runaway you can trigger by pulling the clusters apart.

Now, the math

The probability of the positive class is a sigmoid of the linear score:

P(y=1)=σ(w⊤x+b)=11+e−(w⊤x+b)P(y{=}1) = \sigma(w^\top x + b) = \frac{1}{1 + e^{-(w^\top x + b)}}
σ\sigma
the sigmoid — squashes any score into (0, 1).
w⊤x+bw^\top x + b
the linear score; where it is 0 the probability is 0.5.
▸ Show the derivation

Logistic regression minimizes the cross-entropy (log) loss by gradient descent. On separable data that loss has no finite minimum — pushing ‖w‖ toward infinity drives every predicted probability to 0 or 1, so the weights never settle. Regularization (as in ridge) is what tames it in practice.

Trace it by hand

Take one point x = (1.0, 1.0) with weights w = (0.5, 0.3) and bias b = 0. We push its linear score through the sigmoid to get a probability. Values rounded to 4 decimal places.

  1. Compute the linear score

    w⊤x+b=(0.5)(1.0)+(0.3)(1.0)+0=0.8w^\top x + b = (0.5)(1.0) + (0.3)(1.0) + 0 = 0.8

  2. Squash it through the sigmoid

    σ(0.8)=11+e−0.8=11+0.4493=0.6900\sigma(0.8) = \frac{1}{1 + e^{-0.8}} = \frac{1}{1 + 0.4493} = 0.6900

    e to the minus 0.8 is 0.4493, so the denominator is 1.4493.

  3. Read off the decision

    P(y=1)=0.6900≥0.5  ⇒  y^=1P(y{=}1) = 0.6900 \ge 0.5 \;\Rightarrow\; \hat{y} = 1

  4. Check a point on the other side

    w⊤x′+b=(0.5)(−1.0)+(0.3)(0)+0=−0.5,σ(−0.5)=0.3775⇒y^=0w^\top x' + b = (0.5)(-1.0) + (0.3)(0) + 0 = -0.5, \qquad \sigma(-0.5) = 0.3775 \Rightarrow \hat{y} = 0

    A negative score always lands below probability 0.5 — the boundary is exactly where the score is 0.

What just happened: A score of plus 0.8 became probability 0.69 and a score of minus 0.5 became 0.38 — the sigmoid maps the signed distance from the boundary into a confidence, and the 0.5 cutoff is the score-zero line.

Now Break It

Try this: On non-linearly-separable data, no straight boundary can separate the classes — errors are unavoidable.

Control: Switch dataset to XOR / concentric pattern

What happens: Not linearly separable! No straight line can split these classes — logistic regression is fundamentally limited here.

Where logistic regression is used

Logistic regression is one of the most widely deployed classifiers in industry precisely because it is fast, interpretable, and outputs calibrated probabilities. Banks use it for credit scoring, where each coefficient shows how much a factor like income or existing debt shifts the odds of default. Hospitals use it to estimate the probability that a patient has a condition given lab values, and epidemiologists rely on it to quantify how a risk factor changes disease likelihood. Email spam filters, click-through-rate prediction in online advertising, and A/B test analysis all lean on the same S-shaped model. Its popularity endures because a trained logistic regression can be inspected, audited, and explained to a regulator far more easily than a deep network.

A frequent misconception is that logistic regression performs regression in the usual sense. It is a classification method: it models the probability of a class using the logistic (sigmoid) function, then a threshold turns that probability into a label. Another pitfall is assuming the decision boundary can be curved. Logistic regression draws a linear boundary in the feature space you give it, so genuinely non-linear patterns need engineered features or interaction terms. People also misread the coefficients as effects on probability, when they are actually effects on the log-odds; the same coefficient shifts probability by different amounts depending on where you are on the curve. Finally, the default 0.5 threshold is a convention, not a law, and should be tuned to the costs of each error type.

Frequently asked questions

Why is it called regression if it does classification?
The name comes from its mathematical form: it regresses the log-odds of the outcome onto a linear combination of the inputs, just like linear regression fits a line. The difference is that the output is passed through the logistic function to produce a probability between 0 and 1. That probability is then thresholded to make a class decision, which is why it is used as a classifier.
What does the sigmoid function actually do here?
The sigmoid, or logistic function, takes any real number and squashes it into the range 0 to 1, giving an S-shaped curve. Large positive inputs map close to 1, large negative inputs map close to 0, and 0 maps to exactly 0.5. This lets the model turn an unbounded linear score into a valid probability.
How do I interpret the coefficients?
Each coefficient represents the change in the log-odds of the positive class for a one-unit increase in that feature, holding the others fixed. Exponentiating a coefficient gives an odds ratio, which is often easier to communicate. A positive coefficient increases the predicted probability and a negative one decreases it.
Can logistic regression handle more than two classes?
Yes. The multiclass extension is called multinomial logistic regression, or softmax regression, which produces a probability for each class that all sum to one. An alternative is the one-vs-rest strategy, where you train one binary logistic model per class and pick the highest score.
Does logistic regression need feature scaling?
The model itself does not require scaling to be correct, but scaling helps in practice. Gradient-based optimizers converge faster when features share a similar range, and any regularization penalty is applied more fairly when features are standardized. If you use L1 or L2 regularization, scaling is strongly recommended.
← Back to all visualizationsContinue on the Learning Path →

Written & reviewed by the ML Visualization team · Last updated .