The Decision Threshold
The Decision Threshold — A classifier outputs scores; the decision threshold is the cut-off that turns a score into a class. Moving it trades false positives against false negatives, and the optimal threshold depends on the relative cost of each error, not a default of 0.5.
Frame it as a medical test: two overlapping score distributions and a draggable threshold. Turn the dial on how bad a miss is, and the optimal cut-off slides to match — proving 0.5 is rarely the right answer.
- Healthy (−)
- Disease (+)
- Your threshold
- Optimal
A medical test: raise the cost of missing a disease and the optimal threshold (dashed) slides lower to catch more cases. 0.5 is rarely the right answer.
A medical test: raise the cost of missing a disease and the optimal threshold (dashed) slides lower to catch more cases. 0.5 is rarely the right answer.
The idea in plain words
A classifier outputs a score; the threshold turns that score into a decision. The default 0.5 is rarely right — the best cut-off depends on the consequences. Frame it as a medical test: missing a disease (a false negative) is usually far costlier than a false alarm.
Turn up the cost of a miss and the optimal threshold slides lower to catch more cases. It’s the same TP/FP/FN/TN tradeoff from the confusion matrix, now weighted by what each error actually costs.
Now, the math
Pick the threshold that minimizes expected cost, not error count:
- the cost of a false positive (false alarm).
- the cost of a false negative (a miss).
▸ Show the derivation
The optimal threshold is where the marginal cost of catching one more positive equals the marginal cost of the false positives it admits. As the miss cost rises, that balance point shifts down, so the model flags more cases — exactly what a screening test wants.
Now Break It
Try this: Push the threshold to an extreme — catch everything (all positives) or nothing.
Control: Threshold slider (drag to either extreme)
Last updated .