University of Michigan, Fall 2009

Instructor: Clayton Scott

Classroom: 1690 CSE

Time: MW 10:30-12

Office: 4433 EECS

Email:

Office hours: Monday 2-4 PM

GSI: Gowtham Bellala

Uniqname: gowtham

Office hours: TBA.

__Required text__: None.

__Recommended texts__:

- Duda, Hart, and Stork,
*Pattern Classification*, Wiley, 2001 - Hastie, Tibshirani, and Friedman,
*The Elements of Statistical Learning: Data Mining, Inference, and Prediction*, Springer, 2001 - Bishop,
*Pattern Recognition and Machine Learning*, Springer, 2006 - Sutton and Barto,
*Reinforcement Learning: An Introduction*, MIT Press, 1998

__Additional references__

- Tan, Steinback, and Kumar,
*Introduction to Data Mining*, Addison-Wesley, 2005. - Scholkopf and Smola,
*Learning with Kernels*, MIT Press, 2002 - Mardia, Kent, and Bibby,
*Multivariate Analysis*, Academic Press, 1979 (good for PCA, MDS, and factor analysis). - Rasmussen and Williams,
*Gaussian Processes for Machine Learning*, MIT Press, 2006. - Boyd and Vandenberghe,
*Convex Optimization*, Cambridge University Press, 2004 - MacKay,
*Information Theory, Inference, and Learning Algorithms*, Cambridge University Press, 2003

__Prerequisites__: (the current formal prerequisite is currently
listed as EECS 492, Artificial Intelligence, but this is inaccurate)

- Probability: random variables, densities and mass functions, expectation, independence, conditional distributions, Bayes rule, maximum likelihood estimation, the multivariate normal distribution
- Linear algebra: rank, nullity, linear independence, inner products, orthogonality, positive (semi) definite matrices, eigenvalue decompositions, least squares, pseudo-inverses, projections.

__Lecture notes__:

- Least squares linear regression
- The Bayes classifier
- Linear discriminant analysis
- Logistic regression
- Naive Bayes
- Separating hyperplanes
- Locally linear regression
- Regularization
- Inner product kernels and the kernel trick
- Kernel ridge regression
- Constrained optimization
- Support vector machines

- Principal component analysis
- K-means clustering
- The EM algorithm for Gaussian mixture models
- Kernel density estimation

- Reinforcement learning
- Markov decision processes
- Optimal policies
- Learning policies from experience
- Value function approximation

- Model selection and error estimation
- Feature selection
- Multidimensional scaling
- Nonlinear dimensionality reduction
- Decision trees
- Ensemble methods
- Boosting
- Hierarchical clustering
- Spectral clustering
- Neural networks
- Learning theory
__Novelty detection__- Gaussian process regression

__Grading__:

Homework: 35%

Exam: 10%, Thursday Nov. 12, 6-9 PM, location TBA

Participation and attendance: 5%

Final project: 50%

__Homeworks__:

Homework will be assigned every one or two weeks. The assignments will be
much smaller and easier toward the end of the course, when you are working
on your project.

__Computer programming__

Most or all assignments will involve some computer programming.
MATLAB will serve as the official programming language of the course. You
are free to use another language, such as R, but I will sometimes provide
you with fragments of code, or suggested commands, in MATLAB.

__Group work__:

Group work will take place on two levels. You will work on homeworks in
*small groups* of 2, and the final project in *large groups* of
3 or 4. I will help you find groups as needed.

__Exam__: Time and location TBD.

Collaboration of any form will not be allowed. Allowed materials will be
specified in advance of the exam.

__Final Project__:

There will be a final, open-ended group project. The project must explore
a methodology or application (and preferably both) not covered in the
lectures. The work must
not simply reproduce the results of a paper, but explore some new aspect
of a problem. I will assist groups in selecting a topic as much as
necessary. The project will be judged based on clarity, thoroughness, and
originality. The project will be graded based on the following components:

- Project proposal (10%): Due date TBA. 2 pages max. Each group must meet with me in advance of the proposal deadline to discuss the project.
- Progress report (20%): Due Nov. 25 in class. 5 pages max.
- Final report (35%): Due Thursday, Dec. 17, at noon. 10 to 12 pages.
- Poster presentation (35%): Saturday, Dec 19, 1-5 PM, Duderstadt center room 1180.

__Collaboration__:

Each group will turn in one product representative of the group.
Solutions to homework problems from outside sources may not be used.

__Honor Code__

All undergraduate and graduate students are expected to abide by the
College of Engineering Honor Code as stated in the Student Handbook and
the Honor Code Pamphlet.

__Students with Disabilities__

Any student with a documented disability needing academic adjustments or
accommodations is requested to speak with me during the first two weeks of
class. All discussions will remain confidential.