[EECS 598 - Fall 2015] Theoretical Foundations of Machine Learning |
| Course Info | EECS 598-005, Fall 2015, 3 Credits |
| Instructor | Jacob Abernethy Office: 3765 BBB, Email: jabernet_at_umich_dot_edu |
| Time, Place | TuTh 3:00-4:30pm, 1005 DOW |
| Office Hours | Wednesdays 1:30-3pm |
This course will study theoretical aspects of prediction problems, where we seek to understand themathematical underpinnings of machine learning. A primary objective of the class is to bring students to the frontiers of research in this area. The course will cover, among other things, concentration inqualities, uniform deviation bounds, Vapnik-Chervonenkis Theory, Rademacher Complexity, margin bounds, results for kernel methods, boosting, generalization bounds for artificial neural networks, online learning theory and regret minimization. Along the way, we will dive into several related topics, including connections to convex optimization, minimax equilibrium in games, multi-armed bandit problems, calibration, sequential portfolio selection, option pricing, and differential privacy.
Prerequisites: Familiarity with the analysis of algorithms, probabilistic analysis, and several similar topics. EECS 545 (Machine Learning) will be quite helpful but not strictly necessary. The material is going to be about 90% "theory" and thus potential students must have a strong mathematical background. We shall rely heavily on techniques from calculus, probability, and convex analysis, but many tools will be presented in lecture.
Coursework: There will be 5 problem sets, and the final project for the course will consist of the option to do independent research or to give a literature review presentation to the class.
Grade Breakdown
| 50% for Homeworks | There will be 5 problem sets through the semester |
| 40% for Final Project | Students can do a final project on reviewing some research paper, doing novel research, or implementing some algorithms in an interesting way. More details on this to come. |
| 10% for Participation | Students must scribe 1-2 lectures, participate in class, and can receive participation credit for answering some challenge questions. I will try to make enough opportunities for this. |
Roughly half of the course will follow material from the following text:
There is another text that has a few chapters I would like to cover:
In the last several years, several surveys have come out that explore several topics that we shall cover. I will link to them here, and will mention them in various lectures when appropriate: