Schedule

Lecture Date Topic Materials
Lec. 1Wed, Jan 20Introduction
About the course
Unsupervised Learning
Generative models
Lec. 2Mon, Jan 25Variational autoencoders
Lec. 3Wed, Jan 27Energy-based models
Lec. 4Mon, Feb 1GANs
Lec. 5Wed, Feb 3Autoregressive models
Lec. 6Mon, Feb 8Contrastive learning
Learning from video and non-visual modalities
Lec. 7Wed, Feb 10Language & vision
Lec. 8Mon, Feb 15Visual forecasting
Lec. 9Wed, Feb 173D reconstruction
Lec. 10Mon, Feb 22Motion
Wed, Feb 24No class
Lec. 11Mon, Mar 1Sound & vision
Other unsupervised learning methods
Lec. 12Wed, Mar 3Normalizing flows
Lec. 13Mon, Mar 8Pretext tasks
Advances in deep learning
Lec. 14Wed, Mar 10Attention
Lec. 15Mon, Mar 15Optimization
Learning with less supervision
Wed, Mar 17No class
Lec. 16Mon, Mar 22Gradient-based meta-learning
Lec. 17Wed, Mar 24Non-gradient-based meta-learning
Lec. 18Mon, Mar 29Semi-supervised learning
Lec. 19Wed, Mar 31Weakly-supervised learning
Embodied vision
Lec. 20Mon, Apr 5Reinforcement learning
Lec. 21Wed, Apr 7Robotics
Lec. 22Mon, Apr 12Exploration
Lec. 23Wed, Apr 14Human vision
Lec. 24Mon, Apr 19Project presentations
Lec. 25Wed, Apr 21Project presentations


Staff & Office Hours


Instructor
GSI
Name Office hours time
Andrew Owens Monday 4:45 - 5:30pm
Xixi Hu Thursday 4:00 - 4:45pm
Office hours will take place over video chat, using the same Zoom link as lecture.


Course information

Today's computer vision systems largely rely on supervision from humans, such as object labels, to learn about the world. This course will discuss recent efforts to create methods that avoid the need for this supervision by learning from unlabeled sensory data. Topics will include: deep generative models, self-supervised learning, multimodal models, learning from video, and semi-supervised learning. We will also cover recent advances in deep learning, such as meta-learning and network architectures, that support the goal of unsupervised learning. This is a seminar-based graduate-level class covering very recent advances in unsupervised learning in computer vision. The main focus of the class will be on reading and critiquing recent research papers. In each lecture, students will present and critique several recent research papers. You will also explore these ideas via a self-directed project. While the class is not intended to be an introduction to unsupervised learning or deep learning, we will give a single problem set reviewing some of the core concepts, which will be due approximately 2/3 of the way through the semester.

Lectures: Lectures will take place over Zoom on Monday and Wednesday, 3:00 - 4:30pm. Since this is a discussion-based class, your attendance is required. Missing more than two classes without an excuse will negatively affect your grade. Recordings will only be provided to students enrolled in the class.

Prerequisites: This is an advanced vision course. Students are expected to have taken an introductory vision course before enrolling (EECS 442, 504, or equivalent), so that they will be prepared to read and discuss recent research.

Paper reviews: You'll be required to submit one short paper review each week, beginning the week of Lecture 2. Even though there will be four papers presented each week, you will only be required to submit a review for one of them, of your choice, per week (not per class). Your review should be based on the paper itself, rather than the discussion. It is therefore due before the paper is presented in class (i.e. at 3pm on Monday or Wednesday).

  • Summarize the paper. For most papers, this means explaining technical contributions, such as key mathematical insights, algorithms, and architectures.
  • Briefly explain how the paper relates to previous work, and why its contributions might be (or might not be) important.
  • Summarize the key experiments.
  • Discuss the paper's shortcomings: e.g. limitations to the methods, unconvincing aspects of experiments, presentation issues.

Reviews will be graded as: ✓+, ✓, ✓−, 0. We will not accept late submissions without a valid excuse. However, we will drop your 2 lowest review scores.

Q&A: This course has a Piazza forum, where you can ask public questions. We also appreciate it when you respond to questions from other students! If you have an important question that you would prefer to discuss over email, you may email the course staff (eecs442-fa20-staff@umich.edu), or you can contact the instructor by email directly.

Textbooks: In this class, we'll mostly be reading research papers, rather than textbooks. The following might be useful as reference, though:

If you have feedback for the author of the Szeliski book draft, please submit it here, and we'll pass it along!

Grading: Final grades will be computed as follows:

Final project 45%
Class presentation 25%
Problem set 20%
Participation & reviews 10%

Academic integrity: While you are encouraged to discuss homework assignment with other students, your programming work must be completed individually. You must also write up your solution on your own. You may not search for solutions online, or use existing implementations of the algorithms. Please see the Michigan engineering honor code for more information.

Support: The counseling and psychological services center (CAPS) provides support for a variety of issues, including mental health and stress.

Presentation guidelines

You will be in charge of teaching one class, as part of a group of 3 people (starting Lec. 5). Each class will be organized around a topic of ongoing research. We'll send up a sign-up sheet after the first class, where you will rank

Organization: We suggest organizing most classes as follows:

  1. Background (24 mins)
  2. Paper 1 (24 mins)
  3. Paper 2 (24 mins)
  4. Discussion (8 mins)

The background section is usually the most important part of the class. It should resemble a mini-lecture, covering the "basics" that students will need to understand the paper presentations. For example, if the class is covering papers about variational autoencoders (VAEs), this section should review what a VAE is, and it should touch on any relevant findings that are necessary to understand the papers. Often, this will involve also describing prior attempts to solve the problems that the (much more recent) papers address.

Each paper section should be a critical presentation the work in the paper. You should explain what problem the researchers were addressing, their motivation for what their solution was, and how well they succeeded at that goal. Unlike introductory courses, where methods are largely well-understood and have passed the "test of time", the papers in this class will often have important limitations. We therefore encourage you to take a critical approach to reading the papers, and to describe possible shortcomings. We also encourage you to discuss things in the paper that you do not think were well-justified, and choices by the authors that you did not understand.

Finally, for the (optional) discussion, you will lead a brief interactive session, where students can debate the issues at stake in the papers. For example, you might run a Q&A session where you ask: should we really consider language-based supervision to be "unsupervised", or do we need to interact with the world to learn good representations?. If you'd like the section to be particularly interactive, you can also do this using Zoom breakout sessions. The discussion is optional: if you don't think that it fits in well with the rest of your topic, you may cut it, and spend more time on the rest of the class instead.

Slides: You are allowed to use existing slides and figures, but please clearly credit the authors. Please submit your slides to us in PDF form. By default, we will post your slides only on Canvas, so that they are only visible to those enrolled in the class. Howeve, we'd also be happy to post them publicly if you'd like.

Signing up: We'll assign people to presentation timeslots in two phases (i.e. the first and second halves of the class). You'll fill out a questionaire indicating which classes you'd like to participate in. If you happen to have a group of 3 in mind already, please indicate this on the form, and we will try to assign you to a single topic (we unfortunately cannot accommodate groups with other sizes).

Project guidelines

You'll do a self-directed group project, due at the very end of the course. Groups should be at most 4 students, unless you are given permission from the instructor. Deliverables include:
  • Project proposal (due halfway through the semester).
  • Report (4 pages in CVPR format)
  • Presentation (a 5-min talk)
We'll provide more details as the semester progresses.