Imbalanced data

When accuracy stops meaning anything

Fraud detection, rare disease diagnosis, and manufacturing defect detection share a structural feature: the thing being predicted is rare. If only one in a thousand transactions is fraudulent, a model that predicts “not fraud” for every single transaction achieves 99.9 percent accuracy while being completely useless — it has never once correctly identified the case that actually matters. This is not a hypothetical edge case; it is the default outcome of training naively on imbalanced data without accounting for the imbalance at all, because that trivial always-predict-the-majority model is a genuine local optimum for plain accuracy.

This is exactly why evaluation metrics beyond raw accuracy — precision, recall, and the tradeoff between them — matter enormously once a dataset is imbalanced, and why this lesson sits right before that one.

What to watch for

The video covers the full scope in one pass: why accuracy misleads on imbalanced data, and then a survey of both the resampling and the class-weight fixes, which is exactly the shape of this lesson rather than a narrow dive into one single technique.

Resampling: changing the data

Oversampling duplicates examples of the minority class so it makes up a larger share of the training set, at the risk of the model simply memorising those duplicated examples rather than generalising from them. Undersampling instead removes examples of the majority class, which avoids duplication but throws away data that might otherwise have been useful, and can hurt badly when the original dataset was not large to begin with.

SMOTE (Synthetic Minority Oversampling Technique) improves on plain duplication by generating new, synthetic minority examples rather than copying existing ones — each synthetic point is created somewhere between two real minority examples that are close to each other in feature space. This gives the model new, plausible minority examples to learn from rather than exact repeats, which generally helps it generalise better than duplication alone.

Class weights: changing the cost instead

Rather than touching the data at all, an alternative approach changes what the model is being optimised to minimise. A standard training objective treats every mistake equally; a weighted objective makes a mistake on the minority class cost more than an identical mistake on the majority class, directly countering the model’s natural tendency to ignore a rare class in favour of the much larger one. This has a practical advantage over resampling — the original data is left completely untouched, with no duplicated or synthetic examples introduced — though it requires a model and training setup that actually supports weighted costs, which not every implementation does equally well.

Choosing the right metric is still the real fix

Both of these techniques improve what a model learns, but neither one matters at all if the metric used to judge the model continues to be plain accuracy. Precision, recall, and the balance between them, covered in the evaluation metrics lesson immediately following this one, are what actually reveal whether a model handles the minority class well — and in a genuinely imbalanced problem, that choice of metric usually matters more than which specific resampling technique or weighting scheme gets used.