EECS 545: Machine Learning

University of Michigan, Fall 2014

Instructor: Clayton Scott (clayscot)
Classroom: GG Brown 1504
Time: MW 10:30--12:00
Office: 4433 EECS
Office hours: Mon. 1-4 PM or by appointment
GSI: Takanori Watanabe (takanori)
GSI office hours: Tues. 2-5 PM in EECS 2420

Required text: None.

Recommended texts:

• Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, Second Edition, pdf available for download. This book is also available online to UM users, who may purchase a $25 print version by clicking on the "buy now" link from the online book.
• Mohri, Rostamizadeh, and Talwalkar, Foundations of Machine Learning, MIT Press, 2012, available online to UM users.
• Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
• Duda, Hart, and Stork, Pattern Classification, Wiley, 2001, available online to UM users.
• Sutton and Barto, Reinforcement Learning: An Introduction, MIT Press, 1998, available online to UM users.

Additional references:

• Scholkopf and Smola, Learning with Kernels, MIT Press, 2002, available online to UM users.
• Mardia, Kent, and Bibby, Multivariate Analysis, Academic Press, 1979 (good for PCA, MDS, and factor analysis).
• Boyd and Vandenberghe, Convex Optimization, Cambridge University Press, 2004, pdf available for download, and also available available online to UM users (only 7 users at a time).

Prerequisites: (the current formal prerequisite is currently listed as EECS 492, Artificial Intelligence, but this is inaccurate)

• Probability: jointly distributed random variables, multivariate densities and mass functions, expectation, independence, conditional distributions, Bayes rule, the multivariate normal distribution
• Linear algebra: rank, nullity, linear independence, inner products, orthogonality, positive (semi-) definite matrices, eigenvalue decompositions.

Topics:

• Statistical machine learning
• Unconstrained optimization
• The Bayes classifier
• Linear discriminant analysis
• Naive Bayes
• Logistic regression
• Separating hyperplanes
• Linear regression
• Empirical risk minimization
• Kernels
• Kernel ridge regression
• Constrained optimization
• Support vector machines
• Model selection
• Principal component analysis
• k-means
• Expectation maximization and Gaussian mixture models
• Kernel density estimation
• Spectral clustering
• Multidimensional scaling
• Nonlinear dimensionality reduction
• Feature selection
• The alternating direction method of multipliers
• Decision trees
• Ensemble methods
• Boosting
• Reinforcement learning
• Markov decision processes
• Optimal planning
• Learning policies from experience
• Neural Networks
• Reproducing kernel Hilbert spaces

Grading:
Homework: 45%
Midterm exam: 30%, Thursday Nov. 20, 6-9 PM, location TBA.
Final project: 25%. Project due Friday Dec. 12, reviews due Wed. Dec. 17 at 12 noon.

Homeworks:
Homeworks will be assigned weekly. Applications will be developed through Matlab programming exercises, including face recognition, spam filtering, handwritten digit recognition, image compression, and image segmentation. Most assignments will involve some computer programming. MATLAB will serve as the official programming language of the course. I will sometimes provide you with data, fragments of code, or suggested commands, in MATLAB.

Final Project:
There will be a final project. Groups will be allowed. The project must explore a methodology or application not covered in the lectures. You will be asked to select a paper on a methodology not covered in class, and implement the method. Because of the expected large enrollment, students will assist in grading by reviewing and evaluating other projects in the class.

Collaboration on homeworks:
Each student will prepare the final write-up of his or her homework solutions without reference to any other person or source, aside from the student's own notes or scrap work. Students may consult classmates for the purpose of brainstorming, but not for obtaining the details of solutions. Under no circumstances may you copy solutions or code from a classmate or other source.

Computer use in class:
You may use your computer in class for note taking or note viewing, but otherwise please refrain from using computers or personal electronic devices during class, as these are distracting to me and your classmates.

Honor Code
All undergraduate and graduate students are expected to abide by the College of Engineering Honor Code as stated in the Student Handbook and the Honor Code Pamphlet.

Students with Disabilities
Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential.