I am an Assistant Professor in Computer Science and Engineering at the University of Michigan. My research intersects human-computer interaction (HCI) and applied machine learning (ML), and focuses on sound accessibility. Specifically, I am interested in inventing novel sound sensing and visualization systems to support accessibility and healthcare applications. I completed my PhD from University of Washington, masters from MIT Media Lab, and have worked at Microsoft Research, Google, and Apple.
At University of Michigan, I direct the Accessibility Lab, where we are actively looking for passionate undergraduates, masters, PhD students, postdocs, and collaborators (see below). Our current focus is on sound, with the long-term vision of making sounds and sound information accessible to everyone, everywhere, be it in noisy restaurants when you have trouble understanding a conversation, or at your home when you are wearing your headphones and miss someone knocking. We are accomplishing this vision through designing user-centric systems with an aim to deliver sound information in alternative ways—using visual or haptic feedback or through enhanced audio. A key aspect of this research is the focus on the user—we are conducting a thorough design process to understand our users' concerns, building technology systems to address those concerns, and deploying and studying our built systems with the users in their natural environments.
Our lab publishes work in the most selective HCI venues such as CHI, UIST, and ASSETS. In the past, seven of our papers have been honored with best paper and honorable mention awards. As well, our systems have been publicly launched (e.g., one system has over 100,000 users), have received attention from industries such as Microsoft, Google, and Apple, and are frequently covered by the press.
Nov 15: Invited talk at BostonCHI, the oldest SIGCHI chapter!
Nov 5: Invited talk at Michigan AI Symposium!
Sep 12: Started as an Assistant Professor at the University of Michigan!
Jul 27: Graduated with a PhD from the University of Washington!
May 26: SoundWatch featured in CACM Research Highlights!
Prospective PhD students: I am looking for PhD students with the following set of skills: (1) those with prior HCI and user study experience to lead projects focused on Deaf/disabled population, or (2) those with prior applied ML experience around sound and audio, but who also can build HCI systems and run independent HCI focused projects (through conducting user studies, collecting data, and evaluating end-to-end systems). If you think you fit any of these skills, please send me an email at profdj [at] umich [dot] edu with a brief justification of your skill set (e.g., through relevant research experience), a list of projects in my lab that you are interested in, and your CV. For more details, please read my research and teaching statements.
Undergraduates/Masters students:: Please complete this form and we will reach out to you!
Video of In-Home Sound Awareness System for DHH Users for UW CSE visit day showcase.Read more →
Project Amphibian: a virtual reality scuba diving simulator, my MS thesis at MIT and UIST 2016 paper.Read more →
Video of HoloSound AR Sound Sensing and Feedback for the ASSETS Poster.Read more →
A one-week intense design and innovation workshop that I led in the remote rural area of Kutch, India.View on vimeo →
Our future vision for interactive digitally encoded construction materials, presented at CHI 2016.Read more →
A series of experiments to leverage sensorial nature of "food" for making next-generation digital prototypes.Read more →