EECS 545: Machine Learning

University of Michigan, Fall 2013

Instructor: Clayton Scott (clayscot)
Classroom: FXB 1109
Time: TTh 10:30--12:00
Office: 4433 EECS
Office hours: Mon. 1-4 PM or by appointment
GSI: Robert Vandermeulen (rvdm)
GSI office hours: Tuesday 2-4 PM and Thursday 2-3 PM, EECS 2420

Final Projects from Fall 2011

Final Projects from Fall 2009

Final Projects from Fall 2007

Required text: None. I will share my lecture notes prior to each lecture.

Primary recommended text:

• Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, Second Edition (available online).

Other recommended texts:

• Duda, Hart, and Stork, Pattern Classification, Wiley, 2001.
• Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
• Sutton and Barto, Reinforcement Learning: An Introduction, MIT Press, 1998

Additional references

• Devroye, Gyorfi, and Lugosi, A Probabilistic Theory of Pattern Recognition, Springer, 1996.
• Scholkopf and Smola, Learning with Kernels, MIT Press, 2002
• Mardia, Kent, and Bibby, Multivariate Analysis, Academic Press, 1979 (good for PCA, MDS, and factor analysis).
• Boyd and Vandenberghe, Convex Optimization, Cambridge University Press, 2004

Machine learning bibliography

Prerequisites: (the current formal prerequisite is currently listed as EECS 492, Artificial Intelligence, but this is inaccurate)

• Probability: jointly distributed random variables, multivariate densities and mass functions, expectation, independence, conditional distributions, Bayes rule, the multivariate normal distribution
• Linear algebra: rank, nullity, linear independence, inner products, orthogonality, positive (semi-) definite matrices, eigenvalue decompositions.

Topics:

• Statistical machine learning

• Nearest neighbor classification
• The Bayes classifier
• Linear discriminant analysis
• Logistic regression
• Naive Bayes
• Separating hyperplanes
• Least squares linear regression
• Locally linear regression
• Nonlinear feature maps
• Regularization
• Inner product kernels and the kernel trick
• Kernel ridge regression
• Constrained optimization
• Support vector machines

• Principal component analysis
• K-means clustering
• The EM algorithm for Gaussian mixture models
• Kernel density estimation

• Model selection and error estimation
• Feature selection
• Spectral clustering
• Multidimensional scaling
• Nonlinear dimensionality reduction
• Decision trees
• Ensemble methods
• Boosting
• Hierarchical clustering
• Neural networks
• Gaussian processes
• Reproducing kernel Hilbert spaces
• Multitask and Transfer Learning, Group Sparsity, Matrix Completion, Robust PCA, Multiclass SVM
• Learning theory

Grading:
Homework: 35%
Midterm exam: 30%, Thursday November 7, 6-9 PM, location TBA.
Final project: 35%

Homeworks:
About four or five homeworks will be assigned before the midterm. Applications will be developed through Matlab programming exercises, including face recognition, spam filtering, handwritten digit recognition, image compression, and image segmentation. Most or all assignments will involve some computer programming. MATLAB will serve as the official programming language of the course. I will sometimes provide you with data, fragments of code, or suggested commands, in MATLAB.

Final Project:
There will be a final project. Groups will be allowed. The project must explore a methodology or application not covered in the lectures.  Project guidelines and parameters will be announced at a later date, and may depend on the final enrollment of the course.

Collaboration on homeworks:
Each student will prepare the final write-up of his or her homework solutions without reference to any other person or source, aside from the student's own notes or scrap work. Students may consult classmates for the purpose of brainstorming, but not for obtaining the details of solutions. Under no circumstances may you copy solutions or code from a classmate or other source.

Computer use in class:
You may use your computer in class for note taking or note viewing, but otherwise please refrain from using computers or personal electronic devices during class, as these are distracting to me and your classmates.

Honor Code
All undergraduate and graduate students are expected to abide by the College of Engineering Honor Code as stated in the Student Handbook and the Honor Code Pamphlet.

Students with Disabilities
Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential.