STAT406 - "Elements of Statistical Learning"

Public repository for STAT406 @ UBC - "Elements of Statistical Learning".

LICENSE

The notes in this repository are released under the "Creative Commons Attribution-ShareAlike 4.0 International" license. See the human-readable version here and the real thing here.

Weekly reading and other resources

This is a list of strongly recommended pre-class reading. [JWHT13] and [HTF09] indicate two of the reference books listed below.

• Week 1 (L1): Review of Linear Regression
  • Section 2.1, 2.1.1, 2.1.2, 2.1.3, 2.2, 2.2.1 from [JWHT13]
  • Section 2.4 and 2.6 from [HTF09].
• Week 2 (L2/3): Goodness of Fit vs Prediction error, Cross Validation
  • Section 5.1, 5.1.1, 5.1.2, 5.1.3 from [JWHT13]
  • Section 7.1, 7.2, 7.3, 7.10 from [HTF09].
• Week 3 (L4/5): Correlated predictors, Feature selection, AIC
  • Section 6.1, 6.1.1, 6.1.2, 6.1.3, 6.2 and 6.2.1 from [JWHT13]
  • Section 7.4, 7.5 from [HTF09].
• Week 4 (L6/7): Ridge regression, LASSO, Elastic Net
  • Section 6.2 (complete) from [JWHT13]
  • Section 3.4, 3.8, 3.8.1, 3.8.2 from [HTF09]
• Week 5 (L8/9): Elastic Net, Smoothers (Local regression, Splines)
  • Section 7.1, 7.3, 7.4, 7.5, 7.6 from [JWHT13]
• Week 6 (L10/11): Curse of dimensionality, Regression Trees
  • Section 8.1, 8.1.1, 8.1.3, 8.1.4 from [JWHT13]
• Week 7 (L12/13): Bagging, Classification, LDA, Logistic Regression
  • Section 8.2, 8.2.1, 4.1, 4.2 from [JWHT13]
• Week 8 (L14/15): LDA, LQA, Nearest Neighbours, Trees
  • Section 4.4, 4.3, 2.2.3, 8.1.2 from [JWHT13]
• Week 9 (L16/17): Ensembles, Bagging, Random Forests
  • Section 8.2.1 and 8.2.2 from [JWHT13]
• Week 10 (L18/19): Boosting, Neural Networks?
  • Section 8.2.3 from [JWHT13]
  • Section 10.1 - 10.10 (except 10.7), 11.3 - 11.5, 11.7 from [HTF09]
• Week 11 (L20/21): Unsupervised learning, K-means, model-based clustering
  • Section 10.3 from [JWHT13]
  • Section 13.2, 14.3 from [HTF09]
• Week 12 (L22/23): EM-algorith, Hierarchical clustering
  • Section 10.3 from [JWHT13]
  • Section 8.5, 14.3 from [HTF09]
• Week 13 (L24/25): Principal Components, Multidimensional Scaling
  • Section 10.2 from [JWHT13]
  • Section 14.5.1, 14.8, 14.9 from [HTF09]

Reference books

• [JWHT13]: James, G., Witten, D., Hastie, T. and Tibshirani, R. An Introduction to Statistical Learning. 2013. Springer-Verlag New York

  • Book page, Book PDF

• [HTF09]: Hastie, T., Tibshirani, R. and Friedman, J. The Elements of Statistical Learning. 2009. Second Edition. Springer-Verlag New York

  • Book page, Book PDF

• [MASS]: Venables, W.N. and Ripley, B.D. Modern Applied Statistics with S. 2002. Fourth edition, Springer, New York.

  • Book page, Publisher page