A Single Neuron
Weights, bias, and an activation function in one unit.
Covers: neuron · weights · bias · activation function · weighted sum
Machine learning glossary
Every term below is defined in plain language — and unlike a normal glossary, each one links to an interactive visualization where you can see the idea run on real numbers, drag the data around, and break the algorithm on purpose. If a definition feels abstract, click through and play with it: the concept usually clicks in under a minute.
Definitions are ordered A–Z. Under each entry, the “Covers” line lists the specific skills that concept’s page teaches, so you can jump straight to the visualization that explains the exact thing you’re stuck on.
A
Activation Functions
The nonlinearity that lets networks bend.
Covers: relu · tanh · vanishing gradient · nonlinearity
AdaBoost
Chain weak learners, each fixing the last one’s mistakes.
Covers: adaboost · boosting · weak learners · sample weighting · sequential ensemble
Attention & Transformers
Attention lets each token in a sequence build its representation as a weighted blend of the other tokens, where the weights come from the similarity of learned query and key vectors. It is the core mechanism of the transformer, the architecture behind modern large language models.
Covers: attention · self-attention · transformer · scaled dot-product attention · query key value · large language models
B
Backpropagation
Propagate error gradients backward to update every weight.
Covers: backpropagation · chain rule · gradient · weight update
Bagging
Average many models trained on bootstrap samples.
Covers: bagging · bootstrap aggregating · ensemble · variance reduction
Bias–Variance Tradeoff
Balance underfitting against overfitting.
Covers: bias-variance tradeoff
C
Confusion Matrix
Break predictions into true/false positives and negatives.
Covers: confusion matrix · true positive · false positive · class imbalance · accuracy
Convolution
Convolution slides a small grid of weights (a kernel) across an input, computing a weighted sum at every position. The same kernel is reused everywhere, so it detects one local pattern — an edge, a blob, a texture — wherever it appears in the image.
Covers: convolution · kernel · filter · feature map · weight sharing · stride · padding
Convolutional Neural Networks
A convolutional neural network (CNN) stacks convolution and pooling layers to turn raw pixels into a hierarchy of features — edges, then strokes, then whole shapes — and a final dense layer classifies the result. It is the architecture that made image recognition work.
Covers: convolutional neural network · cnn · pooling · feature maps · image classification · mnist · rotation invariance
Cross-Validation
Rotate the holdout set across k folds for a stable estimate.
Covers: cross-validation · k-fold · model selection · variance of estimate
D
DBSCAN
Cluster by density; label sparse points as noise.
Covers: dbscan · density-based clustering · epsilon · noise points · core points
Decision Boundaries
A decision boundary is the surface a classifier draws to separate classes. Different models draw very different boundaries on the same data: logistic regression a straight line, kNN jagged islands, a tree axis-aligned boxes, an SVM a max-margin street.
Covers: model comparison · non-linear classifiers
Decision Tree
Split the data with a sequence of yes/no questions.
Covers: decision tree · information gain · gini impurity · splitting · tree depth
E
Elastic Net
Blend L1 and L2 penalties to get the best of both.
Covers: elastic net · L1 L2 mixing · regularization · correlated features
Encoding Categorical Features
Turn categories into numbers models can use.
Covers: categorical encoding · one-hot encoding · label encoding
F
Feature Scaling
Put features on the same scale so no one dominates.
Covers: feature scaling · standardization · normalization · z-score
Forward Propagation
Push inputs through the layers to compute a prediction.
Covers: forward propagation · layer computation · matrix multiplication · activations
G
Gaussian Mixture Models (EM)
Soft-cluster data as a blend of Gaussian blobs via EM.
Covers: gaussian mixture model · expectation maximization · soft clustering · latent variables
Gradient Boosting
Fit each new tree to the residual errors of the last.
Covers: gradient boosting · residual fitting · GBDT
Gradient Descent
Gradient descent is an iterative optimization algorithm that minimizes a loss function by repeatedly stepping in the direction of its negative gradient. The learning rate controls the step size and determines whether it converges or diverges.
Covers: gradient descent · learning rate · optimization · convergence · divergence
H
Hierarchical Clustering
Build a tree of clusters by repeatedly merging the closest pair.
Covers: hierarchical clustering · dendrogram · linkage · agglomerative clustering
K
K-Means Clustering
K-means is an unsupervised clustering algorithm that partitions data into k groups by alternating between assigning each point to its nearest centroid and moving each centroid to the mean of its members, minimizing within-cluster variance.
Covers: k-means · clustering · unsupervised learning · local minima · initialization
K-Nearest Neighbors
K-nearest neighbors is a supervised learning algorithm that classifies a point by a majority vote of its k closest labeled examples under a distance metric. It does no training — it simply stores the data and measures distance at prediction time.
Covers: k-nearest neighbors · classification · distance metric
L
Lasso Regression (L1)
Drive some coefficients exactly to zero for feature selection.
Covers: lasso regression · L1 regularization · sparsity · feature selection
Linear Regression
Linear regression is a supervised learning algorithm that fits the best straight line through data to predict a continuous value, choosing the slope and intercept that minimize the squared error (ordinary least squares).
Covers: linear regression · ordinary least squares · residuals · slope · intercept · leverage points
Logistic Regression
Predict class probabilities with an S-shaped curve.
Covers: logistic regression · sigmoid · decision boundary · log loss · linear separability
Loss Functions
A loss function quantifies the disagreement between a model’s predictions and the true values as a single number. Training minimizes this number; the choice of loss (such as MSE or MAE) determines how errors are penalized.
Covers: loss function · mean squared error · mean absolute error · outlier sensitivity
M
Multilayer Perceptron
Stack neurons into layers to learn nonlinear boundaries.
Covers: multilayer perceptron · hidden layers · feedforward network · nonlinear boundary
N
Naive Bayes
Classify using Bayes’ rule and a strong independence assumption.
Covers: naive bayes · bayes theorem · conditional independence · posterior probability
O
Optimizers (SGD · Momentum · Adam)
Optimizers are the update rules that drive gradient descent. Stochastic gradient descent steps on noisy mini-batch gradients; momentum accumulates velocity to power through ravines; Adam adapts a per-parameter step size. They differ most on hard surfaces like ravines and saddles.
Covers: stochastic gradient descent · momentum · adam optimizer · adaptive learning rate · velocity
Overfitting & Underfitting
Underfitting is when a model is too simple to capture the pattern (high error everywhere); overfitting is when it is so flexible it memorizes noise (low train error, high test error). The sweet spot minimizes test error.
Covers: overfitting · underfitting · generalization · model complexity · train-test gap
P
Polynomial Regression
Fit curves by adding polynomial features.
Covers: polynomial regression · feature engineering
Precision, Recall & F1
Trade off catching positives against being right about them.
Covers: precision · recall · f1 score · threshold · precision-recall tradeoff
Principal Component Analysis
Project data onto the directions of greatest variance.
Covers: principal component analysis · variance · eigenvectors · dimensionality reduction · projection
R
Random Forest
Bag decision trees with random feature subsets.
Covers: random forest · feature bagging · decorrelation · out-of-bag error
Ridge Regression (L2)
Shrink coefficients toward zero to reduce variance.
Covers: ridge regression · L2 regularization · coefficient shrinkage · regularization strength
ROC Curves & AUC
Visualize the full threshold tradeoff in one curve.
Covers: roc curve · auc · true positive rate · false positive rate
S
Softmax & Multiclass
Softmax converts a vector of raw class scores (logits) into probabilities that sum to 1 by exponentiating and normalizing. A temperature parameter sharpens it toward a hard argmax or flattens it toward uniform.
Covers: softmax · multiclass classification · temperature · logits · probability
Support Vector Machine
Find the boundary with the widest margin between classes.
Covers: support vector machine · margin · support vectors · hinge loss · regularization C
T
t-SNE
Embed high-dimensional data in 2D preserving local neighborhoods.
Covers: t-sne · perplexity · nonlinear embedding · neighborhood preservation
The Decision Threshold
A classifier outputs scores; the decision threshold is the cut-off that turns a score into a class. Moving it trades false positives against false negatives, and the optimal threshold depends on the relative cost of each error, not a default of 0.5.
Covers: decision threshold · cost-sensitive classification · false negative · operating point
The Kernel Trick
Separate non-linear data by lifting it into higher dimensions.
Covers: kernel trick · RBF kernel · polynomial kernel · feature space · gamma
The Perceptron
The original learning neuron: a linear threshold unit.
Covers: perceptron · linear threshold · perceptron learning rule
Train/Test Split
Hold out data to measure real generalization.
Covers: train test split · holdout set · data leakage
U
UMAP
Fast manifold embedding preserving local and some global structure.
Covers: umap · manifold learning · n_neighbors · min_dist · graph embedding
W
What Is a Model?
A machine learning model is a mathematical function with adjustable parameters that maps inputs to predictions. Training a model means choosing the parameters that best fit observed data.
Covers: machine learning model · prediction · function fitting
Word Embeddings
A word embedding maps each word to a dense vector of numbers so that words used in similar contexts sit close together, and directions in the space capture relationships — famously, king − man + woman lands near queen.
Covers: word embeddings · word2vec · glove · vector arithmetic · cosine similarity · analogy · semantic space