EECS 598: Statistical Learning Theory

University of Michigan, Winter 2014

Instructor: Clayton Scott (clayscot)
Classroom: EECS 1003
Time: MW 9-10:30
Office: 4433 EECS
Office hours: Monday 1-2 or by appt.


Recommended books (on reserve at Engineering library):

Surveys and tutorials:

Lecture notes

  1. Probabilistic setting
  2. The Bayes classifier
  3. Hoeffding's inequality
  4. Empirical risk minimization
  5. Vapnik-Chervonenkis Theory. VC classes, Sauer's lemma, DKW theorem, monotone layers and convex sets.
  6. Sieve Estimators: Consistency and Rates of Convergence
  7. Dyadic Decision Trees
  8. Oracle Inequalities and Adaptive Rates. Structural risk minimization. Adapting to relevant features, intrinsic dimenion.
  9. The Bounded Difference Inequality
  10. Rademacher Complexity. Proof of VC inequality.
  11. Kernels
  12. Reproducing Kernel Hilbert Spaces
  13. Kernel Methods and the Representer Theorem
  14. Calibrated Surrogate Losses
  15. Rademacher Complexity of Kernel Classes
  16. Margin Bounds
  17. Universal Consistency of Support Vector Machines and other Kernel Methods
  18. Rates for Linear SVMs under the Hard Margin Assumption
  19. Kernel Density Estimation
  20. Weakly Supervised Learning, Anomaly detection, classification with label noise.

Homework (40%)
Final report (40%)
Participation (20%)

Homework will be assigned progressively in lecture, and will be due at regular intervals. You will be given at least one week advanced notice of the due date, but I recommend solving the problems as they are assigned, since it will help with your understanding of the lectures.

Final report
Each student will choose one or a few research papers on a particular topic in statistical learning theory, and write a report that summarizes the contributions of the paper(s), including at least a sketch of the main technical ideas. Reports will be evaluated by your peers in a manner that mimics a conference review process. I may be out of town the last week of classes, so the reports may be due slightly before then, such as the last Friday before classes end.

Attendance, classroom interaction, and scribing lecture notes.

Honor Code
All undergraduate and graduate students are expected to abide by the College of Engineering Honor Code as stated in the Student Handbook and the Honor Code Pamphlet.

Students with Disabilities
Any student with a documented disability needing academic adjustments or accommodations is requested to speak with me during the first two weeks of class. All discussions will remain confidential.