Bias-Variance Tradeoff: The U-Shaped Test Error
Model Flexibility:
5
Test MSE at flex =
0.18
← Minimum (optimal)
Training MSE at flex =
0.25
Decomposition
σ² (noise):
0.06
Bias²:
—
Variance:
—
Test MSE (U-curve)
Train MSE
Irreducible σ²
Bias²
Variance
At flex=10:
Train MSE→0 but Test MSE is high — overfitting!
Optimal:
Bottom of the U (flex≈5)