ML Fundamentals

Supervised tasks split into classification (discrete label) and regression (a number). Unsupervised methods include clustering (k-means), dimensionality reduction (PCA, UMAP), and anomaly detection. Self-supervised learning, where the label is derived from the data itself (predict the next word, fill the blank), is how LLMs are pretrained and blurs the first two categories. Semi-supervised and active learning reduce labelling cost by choosing which examples to label.

Framing includes choosing the target variable (what exactly is being predicted), the unit of prediction, the decision the prediction feeds, and the metric that reflects business cost. Data work includes joining sources, handling missing values, deduplication, and defining the point in time at which features are known (to avoid leakage). Training compares several model families with cross-validation. Evaluation is done on a held-out test set and then in a shadow or A/B deployment. Monitoring tracks input drift, prediction drift, and, where available, realised outcomes.

The loss function turns predictions and labels into one number (cross-entropy for classification, squared error for regression). Gradient descent computes the slope of the loss with respect to every weight (via backpropagation in neural networks) and moves the weights against the slope by a step scaled by the learning rate. Epochs are full passes over the data; batches are the chunks processed per step. Regularisation (weight penalties, dropout, early stopping) fights overfitting; hyperparameters are the settings chosen by the engineer rather than learned (learning rate, layer sizes, tree depth).

Bias is error from a model too simple to represent the truth; variance is error from sensitivity to the particular training sample. Expected error decomposes roughly into bias², variance, and irreducible noise. Regularisation (CS229 lecture 8) trades variance for bias by penalising large weights, and cross-validation picks the strength. Ensembles map onto the same axes: bagging (random forests) averages many high-variance trees to cut variance, while boosting adds weak, high-bias learners in sequence to cut bias (lecture 9). Learning theory (the discussion section) gives the sample-complexity intuition: the examples needed to hold variance down grow roughly with the number of parameters, which is why a richer model asks for more data to reach the same validation error.

Time-based splits (train on the past, test on the future) mimic production and expose leakage that random splits hide. Point-in-time correctness means every feature is computed using only data that existed at prediction time. Feature stores centralise those definitions so training and serving compute identical features (avoiding training-serving skew). Drift comes in kinds: input drift (feature distributions move), concept drift (the relationship between inputs and outcome moves), and label delay (you only learn the truth weeks later). Monitoring compares recent input distributions to the training distribution and alerts on divergence.

Ensembles combine many weak models: forests average trees trained on random subsets; boosting trains trees sequentially on the previous errors. SHAP values attribute a prediction to features, giving partial explainability to ensembles. Neural networks generalise the linear model by stacking layers with non-linear activations; convolutional layers suit images, recurrent layers suited sequences until transformers replaced them. Choosing among families is done empirically with cross-validation on the validation set; the winner is rarely predictable in advance, which is why teams try several.

Each layer computes output = activation(W × input + b). Activations (ReLU, GELU) introduce non-linearity; without them, stacked layers collapse to one linear model. Backpropagation applies the chain rule to compute gradients for every weight efficiently. Specialised layers encode structure: convolutions share weights across positions in an image; attention (Module 1, section 3) lets each position in a sequence weigh all others. Mixture-of-experts models route each token through a subset of the network, so a very large model can be cheap per token.

F1 is the harmonic mean of precision and recall, a single number when you need one. The ROC curve plots true-positive rate against false-positive rate across thresholds; AUC summarises it (0.5 is coin-flip, 1.0 is perfect). For imbalanced problems the precision-recall curve is more informative than ROC. Regression models use error metrics (MAE, RMSE) and should be compared to a naive baseline (predict the average, or yesterday's value). Calibration asks whether a predicted 80% actually happens 80% of the time; it matters when the score itself is shown to users or used in expected-value calculations.

Ng's helicopter diagnostic separates two failures that look identical from outside: the optimiser did not find a good solution, or it found a solution that is good by your objective and bad in reality. The test is to score a human expert's behaviour with your own cost function. If the human scores better than the model, the optimiser is failing (train longer, change the algorithm). If the model scores better than the human but flies worse, the objective is wrong and the fix is the metric, not the model. The same test applies to LLM evals and reward models: check whether the answer a human prefers scores higher under your rubric. Error analysis needs ground truth for intermediate stages, which means labelling a small sample per stage; 100 to 200 examples usually suffice. Read the misclassified examples by hand and tally them by cause; the histogram of causes is the roadmap. Lecture 20 applies the same three-way split to reinforcement learning: is it the simulator, the algorithm, or the reward function?

Core components: a feature store (consistent feature definitions for training and serving), an experiment tracker (which data, code, and hyperparameters produced which metrics), a model registry (versioned artefacts with stage: staging, production, archived), a serving layer (batch scoring or a real-time endpoint), and monitoring (input drift, prediction drift, latency, and outcome metrics when labels arrive). Rollouts use shadow mode (score but do not act), canaries, and A/B tests. From the CI/CD source: continuous integration automates build and test on every commit; continuous delivery keeps code deployable at all times; continuous deployment pushes every passing change to production automatically.

LLMs can act as zero-shot classifiers, which makes them ideal for prototyping and for labelling the data a small model later trains on (LLM-assisted labelling). Small language models (under ~10B parameters) fine-tuned on that data often match the LLM on the narrow task at a fraction of latency and cost (ByteByteGo's LLM vs SLM comparison). Embedding models plus a classifier are another cheap middle path for text. The decision should be revisited as prices fall.

Embedding models are trained so that texts with similar meaning have high cosine similarity. Approximate nearest-neighbour indexes (HNSW and similar) make search fast at millions of vectors. Embeddings of different types can share a space (CLIP puts images and captions together), enabling image search by text. Hybrid search combines vectors with keyword scoring so exact terms (product codes, names) still match. Evaluate with recall@k: for known question-passage pairs, how often is the right passage in the top k results?

Prefix caching stores the computed attention state for a shared prompt prefix (your system prompt and few-shot examples) so only the new part is processed. Continuous batching fills the GPU with many sequences at different stages. Speculative decoding lets a small draft model propose several tokens that the large model verifies in one pass; outputs are identical to the large model alone. Flash attention reorganises the attention computation to use GPU memory more efficiently. Quantisation-aware training recovers most of the quality lost by low precision. Whitepaper example: a 2× speed-up for a 2% accuracy drop on a detection task, an acceptable trade in most products.