Skip to content
ML Visualization

Logistic Regression

ClassificationBeginner~7 min

Logistic Regression — Predict class probabilities with an S-shaped curve.

Despite the name, logistic regression is for classification. It squashes a linear score through an S-shaped sigmoid to output a probability between 0 and 1, then draws a decision boundary.

  • Class 0
  • Class 1
  • Boundary (p=0.5)
Sigmoid: score → probability
Dataset
Click action
Training accuracy100%

The idea in plain words

Despite the name, logistic regression classifies. It takes the same linear score as linear regression and squashes it through an S-shaped sigmoid into a probability between 0 and 1. The boundary is where that probability crosses 0.5.

Drag a point and the whole probability field re-forms. If the classes are perfectly separable the fit keeps pushing the weights larger and larger to make the sigmoid ever steeper — a runaway you can trigger by pulling the clusters apart.

Now, the math

The probability of the positive class is a sigmoid of the linear score:

P(y=1)=σ(w⊤x+b)=11+e−(w⊤x+b)P(y{=}1) = \sigma(w^\top x + b) = \frac{1}{1 + e^{-(w^\top x + b)}}
σ\sigma
the sigmoid — squashes any score into (0, 1).
w⊤x+bw^\top x + b
the linear score; where it is 0 the probability is 0.5.
▸ Show the derivation

Logistic regression minimizes the cross-entropy (log) loss by gradient descent. On separable data that loss has no finite minimum — pushing ‖w‖ toward infinity drives every predicted probability to 0 or 1, so the weights never settle. Regularization (as in ridge) is what tames it in practice.

Now Break It

Try this: On non-linearly-separable data, no straight boundary can separate the classes — errors are unavoidable.

Control: Switch dataset to XOR / concentric pattern

← Back to all visualizationsContinue on the Learning Path →

Last updated .