Logistic regression

A classifier wearing a regression’s name

Despite its name, logistic regression is a classification method, used to predict which of two categories something belongs to — whether an email is spam, whether a transaction is fraudulent, whether a patient has a condition. The name survives because the model starts exactly like linear regression, computing a weighted sum of the inputs, and then adds one crucial step that turns that sum into something usable for classification.

Why linear regression’s raw output cannot be used directly

A plain weighted sum of inputs can be any real number — deeply negative, wildly positive, anything in between. Interpreted directly as a probability of belonging to a class, this makes no sense: a probability has to fall between zero and one, and a raw linear output respects no such bound at all.

The sigmoid: squashing a number into a probability

The sigmoid function solves this cleanly. It takes any real number as input and returns a value strictly between zero and one, with large positive inputs mapping close to one, large negative inputs mapping close to zero, and an input of exactly zero mapping to exactly one half. Logistic regression’s full model, then, is: compute the same weighted sum of inputs that linear regression would, and pass the result through the sigmoid to get a genuine probability. A threshold, conventionally 0.5, then turns that probability into a hard yes-or-no classification.

What to watch for

The video builds exactly this chain of reasoning — why linear regression fails for classification, and how the sigmoid fixes it by turning an unbounded number into a bounded probability — from the channel most widely trusted for getting this specific explanation right. It treats the log-loss objective and the decision boundary fairly briefly; both get fuller treatment below.

Why the threshold is a choice, not a given

Converting a probability into a decision requires picking a threshold, and 0.5 is only a default, not a law. Raising the threshold makes the model more conservative — it predicts the positive class only when quite confident, reducing false positives at the cost of missing more true positives. Lowering it does the reverse. Which tradeoff is correct depends entirely on the problem: in a medical screening test, a missed true positive is usually far costlier than a false alarm, which argues for a lower threshold than 0.5, while a spam filter with too low a threshold starts flagging real mail, arguing the other way. This choice connects directly to the precision and recall tradeoff covered in evaluation metrics.

Why log-loss, not squared error

Linear regression trains by minimising squared error, and logistic regression does not — it minimises cross-entropy, often called log-loss in this context. The reason is how each objective behaves near the extremes: squared error treats a prediction of 0.99 when the true label is 1 as only a marginally better prediction than 0.9, while log-loss treats a confident wrong answer as catastrophically costly, penalising a prediction of 0.01 when the true answer is 1 far more severely than squared error would. This sharper penalty near zero and one gives the training process a much stronger, more useful gradient to follow when the model is confidently wrong, which squared error simply does not provide on a probability output.