Regularisation

Inductive Bias

Weight Decay

Shrinkage

Note

TODO derive the regularisation expression for noisy inputs and expected loss.

Consistent Reguraliser

Learning Curve

Early Stopping

Note

Proof for a quadratic error

Double Descent

Parameter Sharing

Hard-Sharing

Soft-Sharing

Note

TODO proof for Gaussian mixture model with prior and posterior

Residual Connections

Model Averaging