Important Update

After four wonderful years at the University of Michigan, I am moving to NYU (and home to NYC) in Fall 2023. I will be jointly appointed between CS in the Courant Institute of Mathematical Sciences and ECE in the Tandon School of Engineering.

You will be redirected to my new page in 5 seconds. If you do not wish to wait, please click the link to my new home page

David Fouhey
Quick info: CV   Google Scholar  
Email (Reveal): xdgtnezstcd.gbhlt Joining my group (please read before emailing)
Physical Location: Beyster (BBB) 3777, University of Michigan
Name FAQ: `foe'-`eee'. It rhymes with snowy or Joey: the key is to forget how it is spelled. It (but not me) is from County Cork, Ireland.
Photos: One picture is hard to identify a person with. Here are some more (but dated).
Summary: I am an Assistant Professor of Computer Science and Engineering at The University of Michigan. I was previously a postdoctoral fellow at UC Berkeley, working with Alyosha Efros and Jitendra Malik. Before that, I received a Ph.D. in robotics from CMU where I worked with Abhinav Gupta and Martial Hebert.

For more information about computer vision at Michigan, see here. For more information about joining my group, please visit here. Please note that I am not taking any Ph.D. students in the December 2022 application cycle (i.e., for Fall 2023) for general computer vision. I am looking to take on one student in computer vision for solar physics. In addition to a background in both physics and machine learning, past experience in any of the following would be of strong interest: Stokes or DEM inversion problems, handling UV/EUV data, or MHD models.

Executive Summary


Research: I work on learning-based computer vision. I want to develop autonomous systems that can learn to build representations of the underlying state and dynamics of the world through observation. I am particularly interested in the following three areas: Teaching and mentoring: I've been able to work on these projects via collaboration with a wonderful group of student collaborators, ranging from the PhD to the BS level. In addition to current students, I'm proud of a large number of MS and BS students now off doing a wide variety of great things in both industry and academia. As a classroom instructor, I've been able to share computer vision with over 1,000 students over the past few years, including over 925 undergraduate students.

Service and outreach: In addition to internal service and one-off outreach efforts, I have run UM's AI4All Summer Program since 2019, which has taught 100 high schoolers over the past four years. I also regularly serve the vision community as an area chair for conferences (CVPR 2019 — 2022, NeurIPS 2020 — 2022, ICLR 2021, ECCV 2022, among other) and as an action editor for TMLR.

I've also provided my expertise for the public interest, for example doing pro-bono work for the Michigan Innocence Clinic. My findings helped exonerate a man who was wrongfully convicted of first degree murder and who was serving a sentence of life without parole.

Funding: These endeavours have been generously supported by both public and private funding, including a NSF CAREER Award (2022). My group's work has been sponsored by The National Science Foundation, DARPA, Toyota Research Institute, The Procter & Gamble Company, Nokia Networks, NASA, and the Lockheed Martin Solar and Astrophysics Laboratory as well as the University of Michigan's Precision Health Initiative and the Michigan Institute for Data Science.

Student Collaborators


Postdoc:
Current PhD student collaborators:

Past Student Collaborators


I am proud of my past student collaborators, now off doing great things elsewhere!

MS students/Visitors
  •   Yinwei Dai(UM CSE BS+MS)Next: PhD Student, Princeton
  •   Max Hamilton(UM CSE BS+MS)Next: Andrew Owen's Group, then PhD Student, UMass Amherst
  •   Zhaoheng Zheng(UM CSE MS)Next: PhD Student, USC CS
  •   Vihang Agarwal(UM ECE MS)Next: Research Asistant, UM Medicine
  •   Ravi Sundaram(UM ECE MS)Next: R&D Research Engineer, Playstation
BS students/Visitors
  •   Siyi Chen(UM CSE BSE)Next: PhD Student, University of Michigan ECE
  •   Dichang Zhang(UM CSE BSE)Next: PhD Student, Stony Brook University CS
  •   Samir Agarwala(UM CSE BSE)Next: MS Student, Stanford CS
  •   Ruiyu Li(UM CSE BSE)Next: MS Student, CMU MLD
  •   Malet Haile(AAU BS)
  •   Tibebu Wassie(AAU BS)
  •   Zhizhuo Zhou(UM CSE BSE)Next: MS Student, CMU Robotics, NSF GRFP Winner
  •   Gemmechu Mohammed(AAU BS)Next: PhD Student, Cornell University CS
  •   Alexander Raistrick(UM CSE BSE)Next: PhD Student, Princeton CS
  •   Xiao Song(UM CSE BS)Next: MEng Student, UC Berkeley CS
  •   Justin Bi(UM CSE BS)
  •   Yige Kristina Liu(UM CSE BS)Next: MS Student, Stanford CS
  •   Michelle Shu(JHU CS BS)Next: PhD Student, Cornell CS
  •   Jiaqi Geng(UM CSE BSE)Next: MS Student, CMU Robotics
  •   Qichen Fu(UM CSE BSE)Next: MS Student, CMU Robotics
  •   Yue Wu(UM CSE BSE)
  •   Zhengyuan Dong(UM CSE BSE)

AAU: Addis Ababa University; JHU: Johns Hopkins University

Teaching

University of Michigan:

Selected Publications

2023
Richard Higgins, David Fouhey
MOVES: Moving Objects in Video Enable Segmentation
CVPR 2023
Disagreement with a really simple background model provides surprisingly effective pseudolabel cues for performing grouping and hand-object association
Nilesh Kulkarni, Linyi Jin, Justin Johnson, David Fouhey
Learning to Predict Scene-Level Implicit 3D from Posed RGBD Data
CVPR 2023
We can learn to predict implicit function-based 3D from posed RGBD.
Linyi Jin, Jianming Zhang, Yannick Hold-Geoffroy, Oliver Wang, Kevin Matzen, Matthew Sticha, David Fouhey
Perspective Fields for Single Image Camera Calibration
CVPR 2023 (Highlight -- 2.5% accept rate)
David F. Fouhey, Richard E.L. Higgins, Spiro K. Antiochos, Graham Barnes, Marc DeRosa, J. Todd Hoeksema, K.D. Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
Large-Scale Spatial Cross-Calibration of Hinode/SOT-SP and SDO/HMI
Accepted in The Astrophysics Journal Supplement Series
[Arxiv]

We fix a more than decade-long issue with pointing and pixel scale in the spectropolarimeter onboard Hinode (which gets cold during eclipse season)
2022
Ahmad Darkhalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard E.L. Higgins, Sanja Fidler, David F. Fouhey, Dima Damen.
EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations
NeurIPS Datasets and Benchmarks 2022
[Paper and Reviews]   [Project Webpage]   [Download]   [Trailer]

A new large-scale dataset of segments of people engaged in interaction with objects, including three new challenges and loads of data.
Chris Rockwell, Justin Johnson, David F. Fouhey
The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs
3DV 2022
[Project Page]   [Paper]   [Bibtex]

Small tweaks let vision transformers imitate much of the 8-pt algorithm, which facilitates learning to estimate full 6D relative camera pose, especially in few-sample settings
Nilesh Kulkarni, Justin Johnson, David F. Fouhey
What's behind the couch? Directed Ray Distance Functions for 3D Scene Reconstruction
ECCV 2022
[Arxiv]   [PDF]   [Project Page]

We can produce high-quality 3D reconstructions from a single RGB image via implicit function by carefully analyzing what we expect networks to produce during training.
Samir Agarwala, Linyi Jin, Chris Rockwell, David F. Fouhey
PlaneFormers: From Sparse View Planes to 3D Reconstruction
ECCV 2022
[Arxiv]   [PDF]   [Project Page]   [Bibtex]

Transformers are really good at integrating evidence across multiple views and producing a planar reconstruction.
Shengyi Qian, Linyi Jin, Chris Rockwell, Siyi Chen, David F. Fouhey
Understanding 3D Object Articulation in Internet Videos
CVPR 2022
[Arxiv]   [Arxiv PDF]   [Bibtex]  

By training on both video data and 3D reconstructions in the right way, we can build models of articulations of 3D objects on ordinary video data.
Brian C. Weeks, Zhizhuo Zhou, Bruce K. O'Brien, Rachel Darling, Morgan Dean, Tiffany Dias, Gemmechu Hassena, Mingyu Zhang, and David F. Fouhey
A deep neural network for high throughput measurement of functional traits on museum skeletal specimens.
Accepted in Methods in Ecology and Evolution.
[Website]  

Bird sizes correlate with temperature. We reduce measurement time of museum specimens by ≈10x, leading to datasets at previously unexplored scales.
Richard E.L. Higgins, David F. Fouhey, Spiro K. Antiochos, Graham Barnes, Mark C.M. Cheung, J. Todd Hoeksema, K.D. Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
SynthIA: A Synthetic Inversion Approximation for the Stokes Vector Fusing SDO and Hinode into a Virtual Observatory
Accepted in The Astrophysics Journal Supplement Series
[Arxiv]   [Open Access]   [Video of SynthIA outputs from May 5 to June 24, 2016]  

Our system produces synthetic solar magnetograms that combine the best aspects of multiple instruments. This system formed the basis of a successful NASA Heliophysics Division Tools and Method grant to integrate the system into SDO/HMI's Joint Science Center.
2021
Dandan Shan*, Richard E.L. Higgins*, David F. Fouhey
COHESIV: Contrastive Object and Hand Embedding Segmentation In Video
NeurIPS 2021
[PDF]   [Bibtex]

By applying the Gestalt principle of common fate at scale, we can learn how to segment hand-held objects with fairly minimal supervision.
Alexander Raistrick, Nilesh Kulkarni, David F. Fouhey
Collision Replay: What Does Bumping Into Things Tell You About Scene Geometry?
BMVC 2021 (Oral)
[PDF]   [Supplement (PDF)]   [Supplement Video (MP4)]   [Bibtex]

Collisions with the world are usually seen as a nuisance. At scale and with a random-walk-inspired formulation, they can be used to learn a depth sensor
Linyi Jin, Shengyi Qian, Andrew Owens, David F. Fouhey
Planar Surface Reconstruction from Sparse Views
ICCV 2021 (Oral)
[Arxiv]   [PDF]   [Project Page]

We can learn to reconstruct scenes from a handful of views with an unknown relationship. Humans seem to do this fine, but it poses serious challenges for computers.
Chris Rockwell, David F. Fouhey, Justin C. Johnson
PixelSynth: Generating a 3D-Consistent Experience from a Single Image
ICCV 2021
[Arxiv [PDF [Project Page]   [Bibtex]   [CSE News Piece]

PixelSynth fuses the complementary strengths of 3D reasoning and autoregressive modeling to create an immersive experience from a single image.
Zhizhuo Zhou, Gemmechu Hassena, Brian C. Weeks, David F. Fouhey
Quantifying Bird Skeletons
CV4Animals Workshop
[PDF]   [Bibtex]  

We can measure bird skeleton specimens extraordinarily accurately and quite fast with deep learning. This system can unlock datasets of birds at unprecedented scales.
Richard E.L. Higgins, David F. Fouhey, Dichang Zhang, Spiro K. Antiochos, Graham Barnes, J. Todd Hoeksema, K.D. Leka, Yang Liu, Peter W. Schuck, Tamas I. Gombosi
Fast and Accurate Emulation of the SDO/HMI Stokes Inversion with Uncertainty Quantification
The Astrophysical Journal (ApJ), Volume 911, Number 2, 2021
[Arxiv]   [Published PDF]   [Bibtex]   [Project Page]   [HMI Nugget]

We can emulate the magnetogram production pipeline of SDO/HMI, a key NASA mission. This system lays the ground-work for SynthIA, which produces best-of-both-worlds style magnetograms
2020
S. Qian*, L. Jin*, D. F. Fouhey
Associative3D: Volumetric Reconstruction from Sparse Views
ECCV 2020
[Arxiv]   [Project Page]   [Code]   [Bibtex]

We can build a voxel-based reconstruction of images from two views, even without access to the relative camera positions
C. Rockwell, D. F. Fouhey
Full-Body Awareness from Partial Observations
ECCV 2020
[Arxiv]   [Project Page]   [Bibtex]   [Code]

Human 3D pose estimation systems work poorly on people as they are usually depicted in video. A self-training method works well at fixing this problem.
S. Jabbour, D.F. Fouhey, E. Kazerooni, M.W. Sjoding, J. Wiens
Deep Learning Applied to Chest X-Rays: Exploiting and Preventing Shortcuts
MLHC 2020
[PDF]   [Bibtex]   [Code]

Deep nets can easily exploit shortcuts (e.g., apparent bone density), but a simple transfer learning approach can help mitigate the use of shortcuts.
D. Shan, J. Geng*, M. Shu*, D.F. Fouhey
Understanding Human Hands in Contact at Internet Scale
CVPR 2020 (Oral)
[PDF]   [Bibtex]   [Project Page & Code]

We built a new dataset and model that enables really accurate recognition of basic hand information. Since hands are key to interaction, this basic information unlocks tons of useful new problems.
M. El Banani, J. Corso, D.F. Fouhey
Novel Object Viewpoint Estimation through Reconstruction Alignment
CVPR 2020
[PDF]   [Supp.]   [Bibtex]   [Code and Project Page]

We can learn to do relative pose estimation by aligning reconstructions
N. Kulkarni, A. Gupta, D.F. Fouhey, S. Tulsiani
Articulation-aware Canonical Surface Mapping
CVPR 2020
[Arxiv]   [PDF]   [Bibtex]

We can build canonical surface maps for objects that articulate, such as elephants and horses
2019
A. Szenicer*, D.F. Fouhey*, A. Munoz-Jaramillo, P.J. Wright, R. Thomas, R. Galvez, M. Jin, M.C.M. Cheung
A Deep Learning Virtual Instrument for Monitoring Extreme UV Solar Spectral Irradiance
Science Advances, Vol. 5, Number 10, 2019
[Open Acess]   [Bibtex]   [Prediction Video]   [Activations Video]   [Overview Video]  

We built a virtual version of the EVE MEGS-A instrument that can serve as a replacement after its electrical short
Press coverage/releases:
inverse.com(Inverse.com)   SETI institute(SETI Institute)
theregister.co.uk(theregister.co.uk)   phys.org(phys.org)
techexplorist(TechExplorist.com)   scientific american(ScientificAmerican.com)
eos.org(Earth & Space Science News)   (hpcwire.com)
(sciencedaily.com)
D. Zhukov, J.-B. Alayrac, G. Cinbis, D.F. Fouhey, I. Laptev, J. Sivic
Cross-task weakly-supervised learning from instructional videos
CVPR 2019
[PDF]   [Project Page]   [Arxiv]   [Bibtex]

By accounting for the compositional nature of language, we can learn better models from instructional videos
R. Galvez*, D.F. Fouhey*, M. Jin, A. Szenicer, A. Munoz-Jaramillo, M.C.M. Cheung, P.J. Wright, M.G. Bobra, Y. Liu, J. Mason, R. Thomas
A Machine Learning Dataset Prepared From the NASA Solar Dynamics Observatory Mission
The Astrophysical Journal Supplement, 242:1, 2019
[PDF]   [Arxiv]   [Bibtex]   [Movie & Explanation]   [Small dataset + demo]

We produced a machine-learning-ready dataset that merges the three instruments aboard the NASA SDO mission
Earlier
A. Kumar, S. Gupta, D. F. Fouhey, S. Levine, J. Malik
Visual Memory for Robust Path Following
NeurIPS 2018 (Oral)
[Project Page]   [PDF]   [Bibtex]  
D. F. Fouhey, W. Kuo, A. A. Efros, J. Malik
From Lifestyle VLOGs to Everyday Interactions
CVPR 2018
[Project Page]   [Arxiv]   [Bibtex]  
S. Tulsiani, S. Gupta, D. F. Fouhey, A. A. Efros, J. Malik
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene
CVPR 2018
[Project Page]   [Arxiv]   [Bibtex]
M. Lescroart, D. F. Fouhey, J. Malik
Convolutional neural networks represent shape dimensions -- but not as accurately as humans
Abstract at VSS 2018
[Abstract]
S. Gupta, D.F. Fouhey, S. Levine, J. Malik
Unifying Map and Landmark Based Representations for Visual Navigation
Arxiv 2017
[Project Page]   [Arxiv]   [Bibtex]
D. F. Fouhey, A. Gupta, A. Zisserman
From Images to 3D Shape Attributes
TPAMI (Pre-print on Arxiv)
The TPAMI version has ugly typesetting (full-width tables on the bottom?) that I was unable to change. Read the Arxiv one.
[Arxiv]   [Bibtex]
R. Girdhar, D. F. Fouhey, M. Rodriguez, A. Gupta
Learning a Predictable and Generative Vector Representation for Objects
ECCV 2016 (Spotlight)
[Publication (PDF)]   [Bibtex]
[Project Page]
D. F. Fouhey
Factoring Scenes into 3D Structure and Style
Doctoral Dissertation
[Dissertation (PDF)]   [Bibtex]
[Defense Slides (PDF)]
D. F. Fouhey, A. Gupta, A. Zisserman
3D Shape Attributes
CVPR 2016 (Oral - Watch the presentation on Youtube)
[Publication (PDF)]   [Bibtex]
[Project Page]   [Poster (PDF)]   [Talk (PPTX)]   [Talk (PDF)]  
R. Girdhar, D. F. Fouhey, K. M. Kitani, A. Gupta, M. Hebert
Cutting through the Clutter: Task-Relevant Features for Image Matching
WACV 2016
[Publication (PDF)]   [Bibtex]
D. F. Fouhey, W. Hussain, A. Gupta, M. Hebert
Single Image 3D Without a Single 3D Image
ICCV 2015
[Publication (PDF)]   [Bibtex]
[Poster (PDF)]   [Supplemental (PDF)]   [Bonus Details (PDF)]
X. Wang, D. F. Fouhey, A. Gupta
Designing Deep Networks for Surface Normal Estimation
CVPR 2015
[Publication (PDF)]   [Bibtex]
D. F. Fouhey, A. Gupta, and M. Hebert
Unfolding an Indoor Origami World
ECCV 2014 (Oral - Watch the presentation on VideoLectures.net)
[Publication (PDF)]   [Bibtex]
[Project Page]   [Extended Results (PDF)]  
D. F. Fouhey and C.L. Zitnick
Predicting Object Dynamics in Scenes
CVPR 2014
[Publication (PDF)]   [Bibtex]
[Poster (PDF)]   [Supplemental (PDF)]
D. F. Fouhey, V. Delaitre, A. Gupta, A. Efros, I. Laptev, and J. Sivic.
People Watching: Human Actions as a Cue for Single View Geometry.
IJCV (extended version of ECCV 2012 paper)
[Preprint (PDF)]   [Final version (via Springer)]
D. F. Fouhey, A. Gupta, and M. Hebert
Data-Driven 3D Primitives for Single Image Understanding
ICCV 2013
[Publication (PDF)]   [Bibtex]
[Project Page]   [Poster (PDF)]  
D. F. Fouhey, V. Delaitre, A. Gupta, A. Efros, I. Laptev, and J. Sivic.
People Watching: Human Actions as a Cue for Single View Geometry.
ECCV 2012 (Oral - Watch the presentation on VideoLectures.net )
[Publication (PDF)]   [Bibtex]
[Project Page]  
V. Delaitre, D. F. Fouhey, I. Laptev, J. Sivic, A. Gupta, and A. Efros.
Scene Semantics from Long-term Observation of People.
ECCV 2012
[Publication (PDF)]   [Bibtex]
[Project Page]
D. F. Fouhey, A. Collet, M. Hebert, and S. Srinivasa
Object Recognition Robust to Imperfect Depth Data.
CDC4CV 2012 Workshop at ECCV 2012
[Publication (PDF)]   [Bibtex]
[Supplemental(PDF)]   [Supp. Video 1]   [Supp. Video 2]  
M. Costanza-Robinson, B. Estabrook, and D. F. Fouhey
Representative elementary volume estimation for porosity, moisture saturation, and air-water interfacial areas in unsaturated porous media: Data quality implications
(Sorry for not posting a pre-print!)
In Water Resources Research, Volume 47, 2011
[Official Version]   [Bibtex]
D. F. Fouhey, D. Scharstein, and A. Briggs.
Multiple plane detection in image pairs using J-linkage.
ICPR 2010
[Publication (PDF)]   [Bibtex]
Implementation (Python and C) [Code (Zip)]   [Poster (PDF)]  

Miscellaneous

You may be interested in the following.

Writing (arranged in chronological order): Miscellaneous

Joke Papers


Sometimes when I feel a creative itch, the end result is a joke publication (which despite the name often have a serious point to make). These are all done with the one and only Daniel Maturana.

Fun & Games