EECS 598: The Ecological Approach (Winter 2020)

This is the tentative schedule.

Date	Topic	Reading Discussed / Discussion Topic	Background Reading	Slides	Assignments/Things Out/Things Due
Week 1 Thu Jan 9	Introduction	None
Week 2 Tue Jan 14	CANCELED	None
Week 2 Thu Jan 16	Self-Supervision 1	(1) Unsupervised Visual Representation Learning by Context Prediction (Doersch et al.) fouhey (2) Representation Learning with Contrastive Predictive Coding (van den Oord et al.) fouhey	(de Sa 1994)		No need to do review
Week 3 Tue Jan 21	Self-Supervision 2	(1) End-to-End Learning of Visual Representations from Uncurated Instructional Videos (Miech et al.) relh (2) A critical analysis of self-supervision, or what we can learn from a single image (Asano et al.) pmitrano	(Held and Hein 1963)
Week 3 Thu Jan 23	Time as Supervision (D)	(1) Learning Correspondence from the Cycle-Consistency of Time (Wang et al.) syqian Discussion on Self-Supervision	(Walk et al. 1988) aka the cats were just bored
Week 4 Tue Jan 28	Project Brainstorming	Project Discussion and Brainstorming
Week 4 Thu Jan 30	Shape and Space 1	(1) Learning Category-Specific Mesh Reconstruction from Image Collections (Kanazawa et al.) jinxinz (2) Unsupervised Learning of Depth and Ego-Motion from Video (Zhou et al.) dandans	(Cutting and Vishton 1995) (Barrow and Tenenbaum 78)
Week 5 Tue Feb 4	Shape and Space 2	(1) CSGNet: Neural Shape Parser for Constructive Solid Geometry (Sharma et al.) mbanani (2) Learning to Infer and Execute 3D Shape Programs (Tian et al.) dongzy	(Koenderink et al. 1996)
Week 5 Thu Feb 6	Shape and Space 3	(1) What Do Single-view 3D Reconstruction Networks Learn? (Tatarchenko et al.) jinlinyi (2) Learning a Multi-View Stereo Machine (Kar et al.) fouhey	(Koenderink 2012)
Week 6 Tue Feb 11	Physics and Dynamics	(1) Modeling Expectation Violation in Intuitive Physics with Coarse Probabilistic Object Representations (Smith et al.) tpower (2) Taking visual motion prediction to new heightfields (Erhardt et al.) natlouis	(Battaglia et al. 2013)		Project Proposals Due
Week 6 Thu Feb 13	Audio and Touch	(1) More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch (Calandra et al.) ahowens (2) Audio-Visual Scene Analysis with Self-Supervised Multisensory Features (Owens and Efros) czyang
Week 7 Tue Feb 18	Affordances and Interaction 1	(1) Holistic Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense (Chen et al.) zeyusun (2) Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Video (Li et al.) strongin	(Grabner et al. 2011)
Week 7 Thu Feb 20	Affordances and Interaction 2	(1) Grounded Human-Object Interaction Hotspots from Video (Nagarajan et al.) hwmiao Discussion on Interaction
Week 8 Tue Feb 25	Object States	(1) Joint Discovery of Object States and Manipulation Actions (Alayrac et al.) rtfeng (2) Category-Level Articulated Object Pose Estimation (Li et al.) pavlasek
Week 8 Thu Feb 27	End2endism	(1) Does computer vision matter for action (Zhou et al.) Discussion on Intermediate Representations	Intelligence without Representation (Brooks 1991)
Week 9 Tue Mar 3	SPRING BREAK	SPRING BREAK
Week 9 Thu Mar 5	SPRING BREAK	SPRING BREAK
Week 10 Tue Mar 10	Simulators 1	(1) Habitat: A Platform for Embodied AI Research (Savva et al.) syqian (2) Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World (Tobin et al.) cravis
Week 10 Thu Mar 12	Canceled	Canceled
Week 11 Tue Mar 17	Simulators 2 (D)	(1) Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias (Gupta et al.) cehusted Discussion on Simulators
Week 11 Thu Mar 19	RL 1	(1) The Paths Perspective on Value Learning (Greydanus and Olah) relh (2) Diversity is All You Need: Learning Skills without a Reward Function (Eysenbach et al.) nileshk	(Lake et al., 2016 – Sections 3, 4)
Week 12 Tue Mar 24	RL 2	(1) Large-Scale Study of Curiosity-Driven Learning (Burda et al.) saeedosa (2) Zero-Shot Visual Imitation (Pathak et al.) choang
Week 12 Thu Mar 26	Imitation (D)	(1) A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (Ross et al.) zhsh Discussion on Learning
Week 13 Tue Mar 31	Lifelong and Otherwise Learning	(1) Continual learning: A comparative study on how to defy forgetting in classification tasks (De Lange et al.) thomaseh (2) Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (Finn et al.) hxixi	(Caruana, 1997)
Week 13 Thu Apr 2	Agents	(1) Supervision via Competition: Robot Adversaries for Learning Tasks (Pinto et al.) datahong (2) Abandoning Objectives: Evolution Through the Search for Novelty Alone (Lehman and Stanley) saeedosa	(Smith et al., 2005)		Project Midterm Report Due
Week 14 Tue Apr 7	Agents	(1) Emergent Complexity via Multi-Agent Competition (Bansal et al.) cnris (2) Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning (Such et al.) lubicz	(Sims, 1994) Video for above
Week 14 Thu Apr 9	Grasping	(1) Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost (Zhu et al.) fouhey (2) Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision (Fang et al.) cxt
Week 15 Tue Apr 14	Navigation 1	(1) Cognitive Mapping and Planning for Visual Navigation (Gupta et al.) alexrais (2) On Evaluation of Embodied Navigation Agents (Anderson et al.) fouhey	(Tolman 1948) (Beware of some dated language)
Week 15 Thu Apr 16	Navigation 2	(1) Bayesian Relational Memory for Semantic Visual Navigation (Wu et al.) kdexd Discussion on Navigation	(Loomis et al. 1993) Map and Plan Baseline
Week 16 Tue Apr 21	Learning from Others	(1) Time-Contrastive Networks: Self-Supervised Learning from Video (Sermanet et al.) cnris (2) Learning Navigation Subroutines from Egocentric Videos (Kumar et al.) fouhey
Week 16 Thu Apr 23	Are we doing things right?	(1) The Bitter Lesson (Sutton) (2) A Better Lesson (Brooks) Discussion	(Newell 1973)
Week 17 Tue Apr 28	Project Presentations	. .			Final Report Due
Week 17 Thu Apr 30	Project Presentations	. .			Project Reviews Due

Legend:

No class	Special
Background -- the world	The world
Background -- agents	Agents

I owe a great debt to Phillip Isola, Saurabh Gupta, and Alyosha Efros for great paper selections that I have lovingly stolen from them. Please take from this reading list as well.