All visualizations

A model is a function fit to data to make predictions.

Loss measures how wrong a prediction is.

Iteratively step downhill on the loss surface.

Race SGD, Momentum, and Adam down the same loss surface.

Watch train and test error form the classic U as complexity grows.

Fit the best straight line through data.

Fit curves by adding polynomial features.

Balance underfitting against overfitting.

Shrink coefficients toward zero to reduce variance.

Drive some coefficients exactly to zero for feature selection.

Blend L1 and L2 penalties to get the best of both.

Predict class probabilities with an S-shaped curve.

Classify by majority vote of nearest neighbors.

One dataset, four classifiers, four boundaries — side by side.

Classify using Bayes’ rule and a strong independence assumption.

Split the data with a sequence of yes/no questions.

Find the boundary with the widest margin between classes.

Separate non-linear data by lifting it into higher dimensions.

Turn raw scores into class probabilities that sum to one.

Average many models trained on bootstrap samples.

Bag decision trees with random feature subsets.

Chain weak learners, each fixing the last one’s mistakes.

Fit each new tree to the residual errors of the last.

Cluster by iterating assign-to-nearest-center then move-center.

Build a tree of clusters by repeatedly merging the closest pair.

Cluster by density; label sparse points as noise.

Soft-cluster data as a blend of Gaussian blobs via EM.

Project data onto the directions of greatest variance.

Embed high-dimensional data in 2D preserving local neighborhoods.

Fast manifold embedding preserving local and some global structure.

Put features on the same scale so no one dominates.

Turn categories into numbers models can use.

Hold out data to measure real generalization.

Rotate the holdout set across k folds for a stable estimate.

Break predictions into true/false positives and negatives.

Trade off catching positives against being right about them.

Visualize the full threshold tradeoff in one curve.

Choosing the cut-off is about consequences, not a fixed 0.5.

The original learning neuron: a linear threshold unit.

Weights, bias, and an activation function in one unit.

The nonlinearity that lets networks bend.

Stack neurons into layers to learn nonlinear boundaries.

Push inputs through the layers to compute a prediction.

Propagate error gradients backward to update every weight.