Machine Learning
Building and training models from data, not calling ones someone else trained. Thirty-three lessons from what learning from data means through classical models, evaluation and neural networks, to where deep learning picks up.
What is machine learning?Types of learningThe ML workflowGeneralization and overfittingExploratory data analysisData cleaningFeature engineeringFeature scaling and encodingImbalanced dataLinear regressionLogistic regressionRegularizationK-nearest neighborsDecision treesSupport vector machinesBagging and random forestsGradient boostingClusteringDimensionality reductionAnomaly detectionEvaluation metricsCross-validationHyperparameter tuningModel selectionNeural network basicsActivation functionsBackpropagationTraining neural networksConvolutional neural networksRecurrent neural networksGenerative modelsAttention and transformersReinforcement learning basics
What learning from data means
- What is machine learning?A system that improves at a task from data rather than being explicitly programmed with rules — and how that differs from using a pretrained model.
- Types of learningSupervised, unsupervised, semi-supervised, self-supervised and reinforcement learning — what distinguishes each, with a concrete example of every one.
- The ML workflowCollect, explore, engineer features, split, train, evaluate, tune, deploy, monitor — and why the real version of this loop is iterative, not a straight line.
- Generalization and overfittingWhy a model that performs well on training data can fail on new data, the bias-variance tradeoff, and why train/validation/test splits exist.
Preparing data a model can learn from
- Exploratory data analysisLooking at data before modeling it — summary statistics, distributions, correlations, and asking what the data actually represents and where it came from.
- Data cleaningMissing values, duplicates, outliers, and inconsistent formatting — finding and handling these, and the judgment calls involved.
- Feature engineeringCreating new input variables from raw data that make patterns easier to learn — transformations, interactions, aggregations and date decomposition.
- Feature scaling and encodingNormalization versus standardization and why it matters for some models and not others; encoding categorical variables into numbers a model can use.
- Imbalanced dataWhen one class vastly outnumbers another, why accuracy becomes a misleading metric, and the techniques — resampling, class weights — that address it.
Models that learn from labelled examples
- Linear regressionFitting a line to predict a continuous value — the cost function, fitting by minimizing squared error, and interpreting the resulting coefficients.
- Logistic regressionA classification method despite the name — the sigmoid function, the decision boundary it creates, and the log-loss objective it is trained against.
- RegularizationL1 (Lasso) and L2 (Ridge) regularization — why unconstrained models overfit, how penalizing large coefficients helps, and the difference between the two.
- K-nearest neighborsA non-parametric method that classifies or predicts from the k closest training examples — choosing k, scaling sensitivity, and prediction-time cost.
- Decision treesSplitting data recursively on feature thresholds to build a tree — how splits are chosen, overfitting via depth, and why a tree is interpretable but unstable.
Combining and reshaping the simple models
- Support vector machinesFinding the maximum-margin hyperplane separating classes, the kernel trick for non-linear data, and the C hyperparameter controlling the tradeoff.
- Bagging and random forestsTraining many models on resampled data and aggregating their predictions — and why random forests also randomize which features each tree can split on.
- Gradient boostingTraining models sequentially, each correcting the ensemble's errors so far, scaled by a learning rate — boosting reduces bias where bagging reduces variance.
Finding structure with no labels at all
- ClusteringGrouping unlabeled data by similarity — k-means and the elbow method, hierarchical clustering and dendrograms, and what "good" means with no labels.
- Dimensionality reductionReducing the number of features while preserving structure — PCA, finding the directions of maximum variance via eigenvectors, and what is lost in the process.
- Anomaly detectionFinding rare, unusual data points with statistical, density and distance-based approaches — and the challenge of validating with few or no labeled anomalies.
Knowing whether a model is actually good
- Evaluation metricsAccuracy, precision, recall and the confusion matrix for classification; MSE, RMSE and R² for regression — and why the right metric depends on the problem.
- Cross-validationWhy a single train/test split can mislead, k-fold and stratified k-fold mechanics, and leave-one-out as the extreme case.
- Hyperparameter tuningParameters versus hyperparameters; grid search, random search and Bayesian optimization — and the risk of overfitting to the validation set itself.
- Model selectionComparing different models fairly using the same validation methodology, and the honest case that a simpler, more interpretable model is often the right choice.
Where deep learning starts
- Neural network basicsThe perceptron and multi-layer networks — neurons, weights and biases — and why stacking layers needs non-linearity to beat linear regression.
- Activation functionsSigmoid, tanh and ReLU — what each does to its input, the vanishing gradient problem, and why ReLU became the default for hidden layers.
- BackpropagationHow gradients of the loss with respect to every weight are computed efficiently via the chain rule, propagating error backward through the network.
- Training neural networksThe practical training loop — forward pass, loss, backward pass, optimizer step, batch size, epochs — and batch normalization and dropout as stabilizers.
- Convolutional neural networksFilters sliding over an image, why this is far more parameter-efficient than a fully-connected layer, pooling, and the conv-pool-connected architecture.
What the simple network grows into
- Recurrent neural networksMaintaining a hidden state updated at each sequence step, the vanishing gradient problem over long sequences, and LSTM/GRU as the fix.
- Generative modelsModels that generate new data rather than just predicting a label — autoencoders, learning a compressed representation, and GANs' adversarial setup.
- Attention and transformersWhy attention let models look at any part of a sequence directly instead of a single bottleneck hidden state, and the architecture built entirely on it.
- Reinforcement learning basicsAn agent learning from an environment's rewards — the agent/environment/state/action/reward loop, explore-vs-exploit, and Q-learning as a concrete example.