About Joining My Group

Long story short, I am always looking for motivated and talented students. I also receive hundreds of emails a day and cannot respond to all inquiries about joining my group. This document is not meant to be unfriendly, but just to save both of us time (many of these things are things I'd say).

• If you are a UM student:
  Contact me by email and let's talk! But to save time, please include your resume, an unofficial transcript, and why you're interested. I don't always have the bandwidth to take on new mentees, but if I don't, I'll try to point you in the right direction of someone who might have the bandwidth. I usually do a good job of responding to these emails and if I haven't, let me know.
  
  
• If you are not in a UM degree program and would like to be my student:
  Please apply directly and mention my name (as well as others that interest you) in your research statement. That's all that is required! Here are answers to common questions:
  • I am always looking for students and will almost always be recruiting.
  • Admissions in anything related to computer vision and machine learning is currently extraordinarily competitive and I cannot assess your chances for you -- it depends highly on my group's needs at the moment and the full pool of applicants. However, you can look at current vision PhD students at Michigan to get a sense of typical profiles.
  • Emailing me genuinely does not improve your chances.
  • I genuinely read all PhD applications that mention my name, but applications are read and decided on by a central committee (I encourage reading Mor Harchol-Balter's guide to graduate school admissions in CS for a good sense of how this process works)
  • I am not involved in the MS process at all
  • I am delighted to talk to people who have been admitted; I cannot talk to people before they are admitted due to time constraints.
  • I cannot reveal information about the process or give status updates.
  • I do not respond to most emails about this.
  
  
• If you are not in a UM degree program and would like to visit:
  I do host visitors occasionally but have obligations to my students first and foremost, as well as to UM students. Here are answers to common questions:
  • I am not looking for visitors without a recommendation from somebody I personally know or know of. If you are currently working with someone, please tell me who.
  • I do not advise capstone projects
  • I do not take any short-term (e.g., less-than-three month) undergraduate visitors except in highly exceptional circumstances.
  • I do not respond to the overwhelming majority of emails about this.

If you've read this page and think you have a question that has not been answered, please feel free to email and put ``(JJ Gibson)'' in the subject line.

News

• Two papers accepted at ECCV. Congratulations to Shengyi, Linyi, and Chris!
  
• Paper accepted at MLHC. Congratulations Sarah!
  
• Three papers accepted at CVPR. Congratulations to Dandan, Mohamed, Nilesh, Jiaqi and Michelle!
  We are also releasing pre-release versions of our model for hand state detection -- contact me if you are interested.
• Our paper on predicting extreme UV solar irradiance is out and getting some press coverage. There are some more down-to-earth things in the pipeline too...

Research

I work on computer vision and machine learning. I want to develop autonomous systems that can learn to build representations of the underlying state and dynamics of the world through observation (and potentially interaction).

Towards this end, I'm particularly interested in two aspects of vision. Both are motivated by the somewhat obvious observation that images are the projection of a real, physical, and 3D world to a 2D image plane of an agent. This has lead to research on understanding the world in terms of:

• physical properties: How do we recover a rich 3D world from a 2D image? I am especially interested in representations -- the answers that are obvious are also obviously defective -- as well as how we should reconcile our strong prior knowledge about this structure of the problem with data-driven techniques.
  
  Lately, I've become interested in applying this more broadly with the hope that we can develop AI systems that can learn how the physical world works from observation, including work on solar physics.
• functional properties: How do we infer and understand opportunities for interaction? I am interested how an agent (e.g., human or robot) can interact with the world, including in terms of what this implies for 3D understanding.

My group's work is sponsored by DARPA, Toyota Research Institute, The Procter & Gamble Company, Nokia Networks, NASA, and Lockheed Martin Solar and Astrophysics as well as the University of Michigan's Precision Health Initiative and the Michigan Institute for Data Science.

Students

A full list of people I work with is here, but my current PhD student collaborators are:

• Richard Higgins, Sep 2019 -- present
• Nilesh Kulkarni (co-advised with Justin Johnson), Sep 2019 -- present
• Shengyi Qian, Sep 2019 -- present
• Liz Olson (co-advised with Emily Mower Provost), Sep 2019 -- present
• Christopher Rockwell (co-advised with Justin Johnson), Sep 2020 -- present; previously CSE MS Student)
• Dandan Shan, Sep 2020 -- present; previously ECE MS Student
• Sarah Jabbour (co-advised with Jenna Wiens), Sep 2020 -- present

My current MS Student collaborators are:

• Linyi Jin (May 2019 -- present)

Teaching

University of Michigan:

• EECS 542 (Advanced Topics in Computer Vision): Fall 2020
• EECS 598 (Special Topics: The Ecological Approach to Visual Perception): Winter 2020
• EECS 442 (Computer Vision): Fall 2019, Winter 2019
• AI4All (a two-week residential program for high schoolers): Summer 2019 -- present

Earlier:

• UC Berkeley 294-43 (Object and Activity Recognition) Spring 2018, Fall 2017, Spring 2017. With Trevor Darrell and Alexei Efros.
• CMU 16-824 (Visual Learning and Recognition), Spring 2016. With Abhinav Gupta.
• Tutorial on 3D Scene Understanding at ECCV 2014 with Abhinav Gupta, Martial Hebert, and Derek Hoiem.
  

Selected Publications

	S. Qian*, L. Jin*, D. F. Fouhey Associative3D: Volumetric Reconstruction from Sparse View ECCV 2020 [PDF Coming Soon]   [Bibtex]
	C. Rockwell, D. F. Fouhey Full-Body Awareness from Partial Observations ECCV 2020 [PDF Coming Soon]   [Bibtex]
	S. Jabbour, E. Kazerooni, D.F. Fouhey, M. Sjoding, J. Wiens Deep Learning Applied to Chest X-Rays: Exploiting and Preventing Shortcuts MLHC 2020 [PDF Coming Soon]   [Bibtex Coming Soon]
	D. Shan, J. Geng*, M. Shu*, D.F. Fouhey Understanding Human Hands in Contact at Internet Scale CVPR 2020 (Oral) [PDF]   [Bibtex]   [Project Page & Code]
	M. El Banani, J. Corso, D.F. Fouhey Novel Object Viewpoint Estimation through Reconstruction Alignment CVPR 2020 [PDF]   [Supp.]   [Bibtex]   [Code and Project Page]
	N. Kulkarni, A. Gupta, D.F. Fouhey, S. Tulsiani Articulation-aware Canonical Surface Mapping CVPR 2020 [Arxiv]   [PDF]   [Bibtex]
	J.Y. Song, J.J.Y. Chung, D.F. Fouhey, W.S. Lasecki C-Reference: Improving 2D to 3D Object Pose EstimationAccuracy via Crowdsourced Joint Object Estimation CSCW (Computer-supported Cooperative Work) 2020 [PDF]   [Bibtex]
	A. Szenicer*, D.F. Fouhey*, A. Munoz-Jaramillo, P.J. Wright, R. Thomas, R. Galvez, M. Jin, M.C.M. Cheung A Deep Learning Virtual Instrument for Monitoring Extreme UV Solar Spectral Irradiance Science Advances, Vol. 5, Number 10, 2019 [Open Acess]   [Bibtex]   [Prediction Video]   [Activations Video]   [Overview Video]
	Press coverage/releases: (Inverse.com)   (SETI Institute) (theregister.co.uk)   (phys.org) (TechExplorist.com)   (ScientificAmerican.com) (Earth & Space Science News)   (hpcwire.com) (sciencedaiy.com)
	D. Zhukov, J.-B. Alayrac, G. Cinbis, D.F. Fouhey, I. Laptev, J. Sivic Cross-task weakly-supervised learning from instructional videos CVPR 2019 [PDF]   [Project Page]   [Arxiv]   [Bibtex]
	R. Galvez*, D.F. Fouhey*, M. Jin, A. Szenicer, A. Munoz-Jaramillo, M.C.M. Cheung, P.J. Wright, M.G. Bobra, Y. Liu, J. Mason, R. Thomas A Machine Learning Dataset Prepared From the NASA Solar Dynamics Observatory Mission The Astrophysical Journal Supplement, 242:1, 2019 [PDF]   [Arxiv]   [Bibtex]   [Movie & Explanation]   [Small dataset + demo]
	A. Kumar, S. Gupta, D. F. Fouhey, S. Levine, J. Malik Visual Memory for Robust Path Following NeurIPS 2018 (Oral) [Project Page]   [PDF]   [Bibtex]
	D. F. Fouhey, W. Kuo, A. A. Efros, J. Malik From Lifestyle VLOGs to Everyday Interactions CVPR 2018 [Project Page]   [Arxiv]   [Bibtex]
	S. Tulsiani, S. Gupta, D. F. Fouhey, A. A. Efros, J. Malik Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene CVPR 2018 [Project Page]   [Arxiv]   [Bibtex]
	M. Lescroart, D. F. Fouhey, J. Malik Convolutional neural networks represent shape dimensions -- but not as accurately as humans Abstract at VSS 2018 [Abstract]
	S. Gupta, D.F. Fouhey, S. Levine, J. Malik Unifying Map and Landmark Based Representations for Visual Navigation Arxiv 2017 [Project Page]   [Arxiv]   [Bibtex]
	D. F. Fouhey, A. Gupta, A. Zisserman From Images to 3D Shape Attributes TPAMI (Pre-print on Arxiv) The TPAMI version has ugly typesetting (full-width tables on the bottom?) that I was unable to change. Read the Arxiv one. [Arxiv]   [Bibtex]
	R. Girdhar, D. F. Fouhey, M. Rodriguez, A. Gupta Learning a Predictable and Generative Vector Representation for Objects ECCV 2016 (Spotlight) [Publication (PDF)]   [Bibtex] [Project Page]
	D. F. Fouhey Factoring Scenes into 3D Structure and Style Doctoral Dissertation [Dissertation (PDF)]   [Bibtex] [Defense Slides (PDF)]
	D. F. Fouhey, A. Gupta, A. Zisserman 3D Shape Attributes CVPR 2016 (Oral - Watch the presentation on Youtube) [Publication (PDF)]   [Bibtex] [Project Page]   [Poster (PDF)]   [Talk (PPTX)]   [Talk (PDF)]
	R. Girdhar, D. F. Fouhey, K. M. Kitani, A. Gupta, M. Hebert Cutting through the Clutter: Task-Relevant Features for Image Matching WACV 2016 [Publication (PDF)]   [Bibtex]
	D. F. Fouhey, W. Hussain, A. Gupta, M. Hebert Single Image 3D Without a Single 3D Image ICCV 2015 [Publication (PDF)]   [Bibtex] [Poster (PDF)]   [Supplemental (PDF)]   [Bonus Details (PDF)]
	X. Wang, D. F. Fouhey, A. Gupta Designing Deep Networks for Surface Normal Estimation CVPR 2015 [Publication (PDF)]   [Bibtex]
	D. F. Fouhey, A. Gupta, and M. Hebert Unfolding an Indoor Origami World ECCV 2014 (Oral - Watch the presentation on VideoLectures.net) [Publication (PDF)]   [Bibtex] [Project Page]   [Extended Results (PDF)]
	D. F. Fouhey and C.L. Zitnick Predicting Object Dynamics in Scenes CVPR 2014 [Publication (PDF)]   [Bibtex] [Poster (PDF)]   [Supplemental (PDF)]
	D. F. Fouhey, V. Delaitre, A. Gupta, A. Efros, I. Laptev, and J. Sivic. People Watching: Human Actions as a Cue for Single View Geometry. IJCV (extended version of ECCV 2012 paper) [Preprint (PDF)]   [Final version (via Springer)]
	D. F. Fouhey, A. Gupta, and M. Hebert Data-Driven 3D Primitives for Single Image Understanding ICCV 2013 [Publication (PDF)]   [Bibtex] [Project Page]   [Poster (PDF)]
	D. F. Fouhey, V. Delaitre, A. Gupta, A. Efros, I. Laptev, and J. Sivic. People Watching: Human Actions as a Cue for Single View Geometry. ECCV 2012 (Oral - Watch the presentation on VideoLectures.net ) [Publication (PDF)]   [Bibtex] [Project Page]
	V. Delaitre, D. F. Fouhey, I. Laptev, J. Sivic, A. Gupta, and A. Efros. Scene Semantics from Long-term Observation of People. ECCV 2012 [Publication (PDF)]   [Bibtex] [Project Page]
	D. F. Fouhey, A. Collet, M. Hebert, and S. Srinivasa Object Recognition Robust to Imperfect Depth Data. CDC4CV 2012 Workshop at ECCV 2012 [Publication (PDF)]   [Bibtex] [Supplemental(PDF)]   [Supp. Video 1]   [Supp. Video 2]
	M. Costanza-Robinson, B. Estabrook, and D. F. Fouhey Representative elementary volume estimation for porosity, moisture saturation, and air-water interfacial areas in unsaturated porous media: Data quality implications (Sorry for not posting a pre-print!) In Water Resources Research, Volume 47, 2011 [Official Version]   [Bibtex]
	D. F. Fouhey, D. Scharstein, and A. Briggs. Multiple plane detection in image pairs using J-linkage. ICPR 2010 [Publication (PDF)]   [Bibtex] Implementation (Python and C) [Code (Zip)]   [Poster (PDF)]

Miscellaneous

You may be interested in the following.

Writing (arranged in chronological order):

• My Ph.D. dissertation, Factoring Scenes into 3D Structure and Style (Dissertation and Defense Slides)
  
• A note on some practical considerations when evaluating surface normals
• My A.B. thesis, Multi-Model Estimation in the Presence of Outliers (Thesis and Poster and Presentation)

Miscellaneous

• CMU RI Thesis Template (zip) based off of a CMU RI tech report template from Daniel Morris.

Joke Papers

Sometimes when I feel a creative itch, the end result is a joke publication (which despite the name often have a serious point to make). These are all done with the one and only Daniel Maturana.

• Keras4Kindergartners.com: Check out our secret backup plan for if research doesn't work out.
• Deep Excel: Everybody knows that deep learning brings about synergy and so does Excel, so Daniel Maturana and I released ExcelNet, a break-through technology that merges the power of Deep Learning with Excel.
  The spreadsheet   The whitepaper   The pitch slides   Protips (actually quite helpful)  
• Visual Rank Estimation: Visually Identifying Rank with Daniel Maturana proves that linear algebra can be replaced with machine learning. It also shows that if you are a CNN, the much-hated jet colormap is actually the best colormap. Winner of the ``Most Frighteningly Like Real Research'' award at SIGBOVIK 2016.
• Cat Basis Purrsuit: I've been told that this is the highlight of my research career and it can only go downhill from here: Cat Basis Pursuit
• Celebrity Learning: You may also know my award-winning work with Daniel Maturana on celebrity-themed learning, making money at home from Hilbert's Nulstellensatz, and more from OneWeirdKernelTrick.com
• Kardashian Kernel: The original, rarely imitated, never duplicated. Originator of the alphabetically-related-work section: The Kardashian Kernel

Fun & Games

• The LaCroix Flavors, Ranked: Uh, yeah
• Venus Transit Poster: I wanted a poster of the transit of venus, and didn't like what I could find. So I made one myself with the original JSOC SDO/AIA data. Here: Highish-res (5k) Higher-res (8k) Padded to 30"x40"
• Paris 20/20: The only way to see Paris is to walk through all 20 arrondissements in a single journey by foot. Co-conspirators (so far!): Mathieu Aubry Andrew Owens.
  Possible variants: here another the originally planned one (and 20 miles too).
• Warning: Deep Nets! As mentioned in the New Yorker! You too can have a warning sign for your door.
• Free parking in Berkeley: Parking people hate him. Find out how to park on Berkeley's campus for free with this 1 weird trick.
• Caffe64: dependency-free deep learning in < 12KB via the magic of assembly. Clone and star it here
• miniml: Everybody these days has a deep learning toolkit that takes forever to download, has tons of dependencies, and produces inscrutable models. In the spirit of simplicity and transparency, I wrote a new package that's only 2172 bytes when compiled, has zero dependencies (not even the C standard library -- that's total bloat), and fits a hidden-layer-free network aka logistic regression. Meet miniml here: miniml .
• Programming in Postscript?: Found in the archives, a postscript file (can.ps)that solves The Missionaries/Cannibals problem, then renders the solution. Send it to a printer, open it in vim, be amazed at how much time I had as an undergrad.
• Revisiting Monet in Light of that Other Painter of light: Monet is so booooring and tranquil. Click here to see him spiced up to look like Thomas Kinkade
• Academic Ancestry and Erdös Number: See where I fit in!
• Sculpture at Berkeley: 3D shapes are fun and Berkeley has a lot of them
• #1 Messiest Desk: My desk was voted #1 messiest desk in the second floor of Smith Hall at Carnegie Mellon during the RI's open house for 2016.
  If a cluttered desk is an indication of a cluttered mind, then what does an empty desk indicate?
• Award-Winning Optimization Sheet: I wrote an award-winning one page cheat-sheet for Convex Optimization at CMU (10-725). Be sure to check the watermark!