What Is Statistical Learning?
Function Approximation
Establishes Y = f(X) + ε and the split between reducible and irreducible error, using two concrete running examples.
Open module →Prediction vs. Inference
The "why estimate f?" fork — a Prediction/Inference toggle on one fitted model that recurs through later modules.
Supervised vs. Unsupervised Learning
A label-visibility toggle that turns the same scatter into a classification task or a clustering problem.
Open module →Regression vs. Classification
Parametric and non-parametric methods trained side by side on the same data: response type, flexibility, and the tradeoffs each brings.
Open module →Assessing Model Performance
What it means to measure P quantitatively: loss vs. performance metric, the standard metrics defined, and the train/test MSE U-curve as flexibility changes.
The Bias-Variance Trade-off
Resampled fits and a live bias²/variance/MSE decomposition — the payoff explaining Module 5's U-shape.
Fitting the Model
Closed-Form Solutions: The Geometry of Least Squares
Why Xβ = y usually has no exact solution, and why the answer is an orthogonal projection, not a compromise.
Gradient Descent
When there's no formula to solve for the minimum: a loss function, a slope, and a small step downhill, repeated until the ground is flat.
Regularization: The Problem of Overfitting
A hypothesis that fits the training data almost perfectly and still fails on new data — why it happens, and how penalizing large coefficients fixes it.