Joyce Y. Chai Professor Computer Science and Engineering 3632 Beyster University of Michigan Ann Arbor, MI 48105 Office phone: 1-734-764-3308 Email: chaijy AT umich.edu Teaching  |  Research  |  Brief Bio   | SLED

My research interests are in the area of natural language processing, situated dialogue agents, human-robot communication, and artificial intelligence. I'm particuarly interested in language processing that is sensorimotor-grounded, pragmatically-rich, and cognitively-motivated. My recent work has focused on grounded language processing to facilitate situated communication with robots and other artificial agents. I'm a member of Michigan AI Lab and direct the Situated Langauge and Embodied Dialogue (SLED) research group . I am also affiliated with Michigan Robotics Institute.

Research

 Selected papers in several topic areas. More information about our research and a full list of papers can be found at the SLED website.

Common Ground and Theory of Mind in Situated Language Communication

• Christian-Paul Bara, Sky CH-Wang, and Joyce Chai. Mindcraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks . Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021. Outstanding Paper Award

  
  

• Shaohua Yang, Qiaozi Gao, Sari Saba-Sadiya, and Joyce Chai. Commonsense Justification for Action Explanation. Conference on Empirical Methods in Natural Language Processing (EMNLP), Brussels, Belgium, November 1-4, 2018.

  
  

• Joyce Chai, Lanbo She, Rui Fang, Spencer Ottarson, Cody Littley, Changsong Liu, and Kenneth Hanson. Collaborative Effort towards Common Ground in Situated Human-Robot Dialogue. The 9th ACM/IEEE Conference on Human-Robot Interaction (HRI), Bielefeld, Germany, March 3-6, 2014. 




• Changsong Liu, Rui Fang, and Joyce Chai. Towards Mediating Shared Perceptual Basis in Situated Dialogue. The 13th Annual SIGDIAL Meeting on Discourse and Dialogue, Seoul, Korea, July 2012. 




Reasoning in Natural Language Understanding

• Shane Storks, Qiaozi Gao, Yichi Zhang, and Joyce Chai. Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understanding . Findings of EMNLP, 2021.

  
  

• Chen Zhang and Joyce Chai. Towards Conversation Entailment: An Empirical Investigation. Conference on Empirical Methods in Natural Language Processing (EMNLP), Cambridge, MA, October, 2010
  




• Matthew Gerber and Joyce Chai. Beyond NomBank: A Study of Implicit Arguments for Nominal Predicates. The 48th Annual Meeting of the Association for Computational Linguistics (ACL). Uppsala, Sweden. July 11-16, 2010. Best Long Paper Award
  




Action and Task Learning through Language Communication and Demonstration

• Yichi Zhang and Joyce Chai. Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring. Findings of ACL 2021.

  
  

• Joyce Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, and Guangyue Xu.  Language to Action: Towards Interactive Task Learning with Physical Agents. Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden, July 13-19, 2018. Invited Paper. Invited talk at ICML/IJCAI/AAMAS, Federated AI Meetings .




• Qiaozi Gao, Shaohua Yang, Joyce Chai, and Lucy Vanderwende.  What Action Causes This? Towards Naive Physical Action-Effect Prediction. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), Melbourne, Australia, July 15-20, 2018.




• Lanbo She and Joyce Chai.  Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), Vancouver, Canada, July 31-August 4, 2017.




• Changsong Liu, Shaohua Yang, Sari Sadiya, Nishan Shukla, Y. He, Song-Chun Zhu, and Joyce Chai. Jointly Learning Grounded Task Structures from Language Instruction and Visual Demonstration. Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, Texas, November 1-5, 2016.




• Qiaozi Gao, Malcolm Doering, Shaohua Yang, and Joyce Chai. Physical Causality of Action Verbs in Grounded Language Understanding. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany, August 7-12, 2016. 
  




• Lanbo She and Joyce Chai. Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany, August 7-12, 2016.   




• Shaohua Yang, Qiaozi Gao, Changsong Liu, Caiming Xiong, Song-Chun Zhu, and Joyce Chai. Grounded Semantic Role Labeling. Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), San Diego, CA, June 12-17, 2016. 
  




• Lanbo She, Shaohua Yang, Yu Cheng, Yunyi Jia, Joyce Chai, and Ning Xi. Back to the Blocks World: Learning New Actions through Situated Human-Robot Dialogue. Proceedings of 15th SIGDIAL Meeting on Discourse and Dialogue, Philadelphia, PA, June 18-20, 2014. 




Collaborative Referential Grounding in Embodied Communication

• R. Fang, M. Doering, and J. Y. Chai. Embodied Collaborative Referring Expression Generation in Situated Human-Robot Dialogue. Proceedings of the 10th ACM/IEEE Conference on Human-Robot Interaction (HRI), Portland, Oregon, March 2-5, 2015.




• Changsong Liu and Joyce Chai. Learning to Mediate Perceptual Differences in Situated Human-Robot Dialogue. Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI), Austin, Texas, January 25-30, 2015. 




• Rui Fang, Malcome Doering, and Joyce Chai. Collaborative Models for Referring Expression Generation towards Situated Dialogue. Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI), Quebec, Canada, August, 2014. 




• C. Liu,  L. She, R. Fang, and J. Y. Chai. Probabilistic Labeling for Efficient Referential Grounding based on Collaborative Discourse. Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics (ACL), Baltimore, MD, June 22-26, 2014. 




• R. Fang, C. Liu, L. She, and J. Y. Chai. Towards Situated Dialogue: Revisiting Referring Expression Generation. Conference on Empirical Methods in Natural Language Processing (EMNLP), Seattle, WA, October, 2013
  




Nonverbal Modalities in Natural Langauge Processing

• Zahar Prasov and Joyce Chai. Eye Gaze with Speech Recognition Hypotheses to Resolve Exophoric References in Situated Dialogue. Conference on Empirical Methods in Natural Language Processing (EMNLP), Cambridge, MA, October, 2010
  




• Shaolin Qu and Joyce Chai. Incorporating Temporal and Semantic Information with Eye Gaze for Automatic Word Acquisition in Multimodal Conversational Systems.. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP).Honolulu, October 2008.
  




• Shaolin Qu and Joyce Chai. An Exploration of Eye Gaze in Spoken Language Processing for Multimodal Conversational Interfaces.. Meeting of the North American Chapter of the Association of Computational Linguistics (NAACL-07), Rochester NY, April, 2007.
  

Recent Invited Talks at Conferences/Workshops November 2021: "Intuitive Physics in Commonsense Language Understanding," the joint 15th Linguistic Annotation Workshop and 3rd Designing Meaning Representations Workshop , EMNLP. October 2021: "Language, Action, and Physical Causality," Reasoning and Interaction Conference , Gothenburg University. June 2021: "Communicative Learning in Language," 2nd Workshop on Data Science with Human-in-the-loop: Language Advances (DaSH-LA) , NAACL. June 2021: "Grounding Language to Perception and Action," 2nd Workshop on Advances in Language and Vision Research (ALVR 2021) , NAACL. July 2020: "Situated Human-Machine Communication," ACL 2020 Workshop on Natural Language Interfaces. Online Meeting. Feb 2020: "Robot Learning through Language Communication, " AAAI Workshop on Intelligent Process Automation , New York, NY. June 2019: "Collaboration in Situated Language Communication, " ICML 2019 Workshop on Imitation, Intent, and Interaction , Long Beach, CA. June 2019: "Collaboration in Situated Language Communication," NAACL 2019 Workshop on Spatial Language Understanding and Grounded Communication for Robotics , Minneapolis, MN. December 2018: "Language Communication with Robots, " NeurIPS 2018 Workshop on Visually Grounded Interaction and Language, Montreal, Canada. August 2018: "Language Communication with Robots," NLPCC 2018, Hohhot, China. July 2018: "Language to Action: Towards Interactive Task Learning with Physical Agents,", AAMAS  / ICML / IJCAI, Federated AI meetings, Stockholm, Sweden. June 2018: RSS 2018 Workshop on Models and Representations for Human-Robot Communication, Pittsburg, PA. November 2017: "New Challenges in Situated Language Processing towards Human-Robot Communication," AAAI Fall Symposium on Natural Communication for Human-Robot Collaboration, Arlington, VA. August 2017: "Teaching Robots New Tasks through Language Instructions," ACL 2017 Workshop on Language Grounding for Robotics, Vancouver, Canada. Nov 2016: "Language Grounding towards Situated Human-Robot Communication," EMNLP 2016 Workshop on Uphill Battles in Language Processing - Scaling Early Achievements to Robust Methods, Austin, TX.