Table of Contents

Topics and Course Structure (top)

The first half of the course will cover the fundamental components that drive modern deep learning systems for computer vision:

  • Linear classifiers
  • Stochastic gradient descent
  • Fully-connected networks
  • Convolutional networks
  • Recurrent networks

In the second half of the course we will discuss applications of deep learning to different problems in computer vision, as well as more emerging topics. During this second half the tone of the course will shift slightly towards a seminar: we will omit some details of the systems we discuss, instead focusing on the core concepts behind those applications. We will touch topics such as:

  • Attention and transformers
  • Object detection
  • Image segmentation
  • Video classification
  • Generative models (GANs, VAEs, autoregressive models)

Lectures (top)

Lectures will be Monday and Wednesday, 4:30pm to 6pm. Lecture attendance is not required, and each lecture will be recorded and posted on the course website.

We will hold lectures remotely over Zoom for the first two weeks (Lectures 1 to 3). See Canvas for the Zoom meeting information.

In-person lectures will be in Chrysler 220.

Office Hours (top)

Office hours are your time to ask questions one-on-one with course staff, and get clarification on concepts from the course. All office hours will be held virtually; you can find the office hour schedule and links to meetings on the Google Calendar.

Course Expectations (top)

Deep Learning is a fast-moving, empirically-driven research field. Much of the content we will cover is taken from research papers published in the last 5 to 10 years. It is also a large and fast-growing field of research: there are thousands of research papers published each year on computer vision, deep learning, and related topics. It is possible that we will discuss some new paper or result that will be published or announced during the semester!

Learning about such a fast-moving field is an exciting opportunity, but covering it in a traditional course setting comes with some caveats you should be aware of.

  • No canonical curriculum: Many topics in mathematics and computer science such as linear algebra, real analysis, discrete mathematics, data structures and algorithms, etc come with well-established curricula; courses on such subjects can be found at most universities, and they tend to cover similar topics in a similar order. This is not the case for emerging research areas like deep learning: the set of topics to be covered, as well as the order and way of thinking about each topic, has not yet been perfected.
  • Few learning materials: There are very few high-quality textbooks or other learning materials that synthesize or explain much of the content we will cover. In many cases the research paper that introduced an idea is the best or only resource for learning about it.
  • Theory lags experiments: At present, deep learning is primarily an empirically driven research field. We may use mathematical notation to describe or communicate our algorithms and ideas, and many techniques are motivated by some mathematical or computational intuition, but in most cases we rely on experiments rather than formal proofs to determine the scenarios where one technique might outpeform another. This can sometimes be unsettling for students, as the question “why does that work?” may not always have a precise, theoretically-grounded answer.
  • Things will change: If you were to study deep learning ten years from now, it is very likely that it will look quite different from today. There may be new fundamental discoveries, or new ways of thinking about things we already know; there may be some ideas we think are important today, that will turn out in retrospect not to have been. As a concrete example, when we first taught a similar course at Stanford in 2015, Batch Normalization and Residual Networks had not yet been discovered; these have since become hugely important, and are used in nearly all practical computer vision systems. There may be similarly impactful results lurking right around the corner.

Prerequisites (top)

The following skills will be useful for success in this course:

  • Programming: You should be comfortable programming in Python. You should be familiar with algorithms and data structures at the level of EECS 281 or equivalent. Familiarity with numpy or similar frameworks for numeric programming will be helpful but is not strictly required.
  • Probability: You should have been exposed to probability distributions, random variables, expectations, etc. at the level of EECS 401, MATH 425, or equivalent.
  • Vector Calculus: You will need to compute gradients of vector-valued functions, so you should be comfortable doing so.
  • Machine Learning: Some familiarity with machine learning (at the level of EECS 445 or equivalent) will be helpful but not required; we will review important concepts that are needed for this course.

Textbooks (top)

There is no required textbook for the course. Some lectures will include suggested reading from “Deep Learning” by Goodfellow, Bengio, and Courville; however there is no need to purchase a copy since it is available for free online.

Piazza (top)

The primary way to communicate with the course staff is through Piazza. If you have questions about course concepts, need help with homework, or have questions about course logistics, you should post on Piazza instead of emailing course staff directly. Since Piazza is a shared discussion forum, asking and answering questions there can benefit other students as well.

However do not post solutions to homework assignments on Piazza. If you have questions about a particular piece of code you have written for an assignment, then you should make a private post on Piazza.

Debugging student code via Piazza can be extremely time-consuming for course staff. If you are posting questions about specific help with your code, you must adhere to the following:

  • You should only post short snippets of code; no more than 20-30 lines
  • You must ask a specific, concrete question about your code – more detailed than “it doens’t work”
  • You must explain what you have tried so far to solve the issue on your own

See the Stack Overflow guidelines for asking good questions. The same guidelines apply to asking good questions on Piazza!

Piazza questions should not be used for last-minute debugging. We will strive to adhere to the following when answering Piazza questions:

  • You should never expect an answer within 30 minutes of posting.
  • From Monday to Friday, 10am - 6pm we will respond to questions within 2 hours.
  • During other times, we will respond to questions within 12 hours.

We will also use Piazza to make announcments about the course, such as the release of new homework assignments, details about the midterm exam, or other logistical issues.

If you need to discuss any sensitive matter that you would prefer not be shared with the entire course staff then you can email the instructor directly. However the vast majority of questions about the course should be asked on Piazza.

Homework (top)

You will complete six programming assignments over the course of the semester. All homework assignments will be in Python, and will use PyTorch on Google Colab.

Exam (top)

There will be a midterm exam covering the content of Lectures 1 to 13. It will consist of true/false, multiple choice, and short answer questions.

Mini-Project (top)

Instead of a final exam, at the end of the semester you will complete a mini-project working in groups of 1 to 3 students.

The homework assignments walk you through implementing things with extensive starter code; in contrast the mini-project will not provide any starter code whatsoever, leaving you to implement an entire deep learning pipeline from scratch.

We will provide several choices of projects to complete, which have been scoped to be feasilbe given your knowledge toward the end of the semester. Each project description will include an overview of the concept to be implemented, links to one or more papers or resources with more details, and concrete deliverables (e.g. achieve performance X using metric Y on dataset Z).

UPDATE: The mini-project is now optional. See the Grading Policy section.

Grading Policy (top)

  • Homework (60%): There will be six programming assignments, each worth 10% of your final grade.
  • Midterm (20%): The midterm exam will be worth 20% of your final grade.
  • Mini-Project: (20%): The mini-project will be worth 20% of your final grade.

UPDATE: As discussed in Lectures 13 and 14, the late release of HW4 caused us to rethink our assignments for the second half of the semester. We decided to give each student the choice of:

  • Skip the mini-project, and submit only HW1-6
  • Skip one homework, and complete a mini-project
  • Complete all homework and the mini-project

To implement this, we will compute each student’s grade using three different schemes; their final grade in the course will be the max over the three schemes.

Scheme A (6/6 HW and no project):

  • HW1-3: 12% each
  • Midterm: 22%
  • HW4-6: 14% each

Scheme B (5/6 HW and project):

  • HW1-3: 12% each
  • Midterm: 22%
  • HW4-6: 14% each
  • Mini-Project: Replaces lowest homework

Scheme C (6/6 HW and project):

  • HW1-6: 10% each
  • Midterm: 20%
  • Mini-Project: 20%

We will use the following base scheme to convert raw scores in the course into letter grades:

  • A+: >= 98%
  • A: >= 92%
  • A-: >= 90%
  • B+: >= 88%
  • B: >= 82%
  • B-: >= 80%
  • C+: >= 78%
  • C: >= 72%
  • C-: >= 70%
  • D+: >= 68%
  • D: >= 62%
  • D-: >= 60%
  • F: < 60%

Depending on the distribution of raw scores at the end of the semester, we may apply a curve; however the final curve will be no stricter than the base scheme. For example, if you receive a raw score of 85% in the course, you will be guaranteed a letter grade of at least B; however you may receive a higher letter grade depending on the overall distribution of raw scores.

Late Policy (top)

Homework assignments will be turned in on autograder.io, and are due at 11:59pm ET on their assigned due date.

Each student has a total of three six free late days to use on assignments throughout the semester. Each late day allows you to turn in a single homework assignment up to 24 hours past the initial deadline with no penalty; there is no restriction on the number of free late days you may use per assignment. You do not need permission to use free late days; simply turn in the assignment when you are done, and we will note that you used a free late day when the graded assignment is returned to you.

Once you have exhausted your three free late days, late work will receive a penalty of 25% per day. Again, there is no need to obtain permission in order to turn in late work – we will apply the penalty when grading, and it will be noted when we return the assignment to you. You cannot receive less than 0% on an assignment no matter how many days late it is.

You may not use late days on A6 or the mini-project.

Example:

  • Assignment 1 is due on Friday at 11:59pm; you turn it in at 3am on Sunday. The assignment is two days late. This exhausts two of your three free late days, and you receive no penalty of the assignment.
  • Assignment 2 is due the next Friday at 11:59pm, and you turn it in at 10pm on Monday. The assignment is three days late. This exhausts your last remaining free late day, so you receive a penalty of 50% on the assignment. We grade the assignment as usual, and suppose you recieve a score of 88/100; after applying the late penalty your final recorded score for the assignment is 38/100.
  • Assignment 3 is due the next Friday at 11:59pm, and you turn it on 5pm on Monday. The assignment is three days late, and you receive a late penalty of 75%. We grade the assignment as usual, and suppose you receive a score of 65/100; after applying the late penalty your final recorded score for the assignment is 0/100.

When determining late days, we will use the last revision time of the entire assignment. This means that if you turn in a partially completed assignment before the due date, and then turn in a revised, completed assignment a day after the due date, the entire assignment will be treated as if it had been turned in a day past the deadline.

We will not grant additional late days except in extreme circumstances (serious illness, family tragedy, etc.) Please see the instructor if such a situation arises. In particular we will not grant additional late due to deadlines in other courses, academic publication deadlines, travel to academic conferences, or other such events that you can plan ahead for at the start of the semester.

Autograder.io has its own late token system, but it does not currently support our precise late policy of accepting late work with penalty once late tokens are exhausted. For this reason we will not use autograder.io to track late tokens; we have granted everyone 1000 late tokens on autograder.io, but we will only use autograder.io to track submission timestamps; the accounting of late tokens will happen outside of autograder.io.

Collaboration Policy (top)

You are encouraged to work in groups and discuss course materials with your classmates. However, all work you submit must be your own.

The following are examples of acceptable collaboration:

  • Forming a study group to discuss and review course concepts
  • Discussing general strategies for solving homework problems
  • Reviewing for exams with other students

The following are examples of unacceptable collaboration:

  • Turning in the same code as another student
  • Looking at another student’s code while writing your own
  • Looking at solutions to similar assignments that you find online