EECS 595 001 FA 2021

Instructor: Joyce Chai (chaijy@umich.edu)

Time: Wednesday 1:30-3:00 pm; Friday 3:00-4:30 pm 

Class Location:  1013 DOW

Course Description

The field of Natural Language Processing (NLP) is primarily concerned with computational models and computer algorithms to process  human languages, for example, automatically interpret, generate, and learn natural language. In the past twenty years, the rise of the world wide web, mobile devices, and social media have created tremendous opportunities for exciting NLP applications. Recent advances in machine learning (e.g., deep learning) have also paved the way to tackle many NLP problems in the real world. This course provides an introduction to the state of the art in modern NLP technologies. In particular, the topics to be discussed include: syntax, semantics, discourse, deep learning for NLP, and their applications in information extraction, machine translation, and dialogue systems.

Text book

Speech and Language Processing, an introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, third edition (draft) (Links to an external site.), by Daniel Jurafsky and James Martin, Prentice Hall  (JM for short).  

Optional: Neural Network Methods for Natural Language Processing, Yoav Goldberg, Synthesis Lectures on Human Language Technologies

Prerequisite

Proficiency in Python programming (programming assignments will be in Python).  Knowledge and experience in machine learning is preferred.  

Course Grades

The work in this course consists of four homework assignments and a final project.  Each assignment may include a written portion and a programming portion. 

- Homework assignments: 60%

- Final Project: 40%  (Proposal: 5%;  Related work: 10%; Presentation: 10%; Final Report: 15%)

Schedule of Topics and Assignments

Date	Topics	Reading and Assignments
Sept 1	Introduction	JM Chapter 2
Sept 2	Language Modeling with N-Grams	JM Chapter 3 HW1 assigned
Sept 8	Text Classification and Sentiment Analysis	JM Chapter 4;
Sept 10	Logistic Regression and Neural Network	JM Chapter 5 & 7
Sept 15	Vector Semantics	JM Chapter 6
Sept 17	Neural Language Model	JM Chapter 7.2 HW1 due, HW2 assigned
Sept 22	POS and Recurrent Neural Networks	JM Chapter 9
Sept 24	Contextual Embedding- Transformers
Sept 29	Constituency Grammar and Parsing	JM Chapter 12
Oct 1	Statistical Parsing	JM Chapter 13,  HW2 due, HW3 assigned
Oct 6	Dependency Parsing	JM Chapter 14
Oct 8	Meaning Representations	JM Chapter 15
Oct 13	Semantic Parsing
Oct 15	Semantic Roles and Selectional Restriction	JM Chapter 19, 20,   HW3 due, HW4 assigned
Oct 20	Coreference Resolution	JM Chapter 21,
Oct 22	Discourse Coherence	JM Chapter 22
Oct 27	Information Extraction	JM Chapter 17
Oct 29	Question Answering	JM Chapter 23. HW4 due
Nov 3	Machine Translation	JM Chapter 11, Final Project Proposal due
Nov 5	Dialogue Systems and Chatbots	JM Chapter 24
Nov 10	Recent Advances (1)
Nov 12	Recent Advances (2)	Final Project Related Work due
Nov 17	Recent Advances (3)
Nov 19	Recent Advances (4)
Nov 24	Thanksgiving Break
Nov 26	Thanksgiving Break
Dec 1	Recent Advances (5)
Dec 3	Recent Advances (6)
Dec 8	Final Project Presentation
Dec 10	Final Project Presentation
Dec 16		Final Project Report Due

All course materials, e.g., recorded videos, slides, assignments are in CANVAS

Course Policies

Homework

Homework must be turned in on the date that it is due, by 11:59 pm. The homework must be submitted electronically using Canvas and we will use the later timestamp to validate turn-in time. It is your responsibility to ensure that the homework has been uploaded successfully by the due date.   Homework that is incorrectly uploaded will be subject to the associated late penalty. Late homework will be penalized 10% per day.  Homework turned in after three days will not be accepted.  

Also, note that any changes you make to the homework already submitted on Canvas count as a resubmission.  If you make any changes to the assignment after the due date has passed you will be assigned a late penalty based on the number of days that have passed.  For example, if you edit an assignment on March 5th and it was due on March 2nd, you will be assigned a 30% penalty (10% per day) as explained above. 

Office Hours

The instructors will have regularly scheduled office hours each week. You are encouraged to make use of these to discuss aspects of the course including lecture material and homework problems. In cases where you cannot make office hours, contact the course staff to arrange an appointment.

Piazza

We have enabled Piazza to facilitate collaborative problem solving between students. It does not serve as constant on-demand access to course instructors.  If you have pressing concerns, make sure to ask during lectures or office hours. Do not post homework solutions on Piazza. 

Academic Honesty

Honor code

All homework submitted must be your own work.  Review the Honor Code at the College of Engineering here: http://www.engin.umich.edu/college/academics/bulletin/rules (Links to an external site.)

The Honor Code is based on these tenets:

1. Engineers must possess personal integrity both as students and as professionals. They must be honorable people to ensure safety, health, fairness, and the proper use of available resources in their undertakings.

2. Students in the College of Engineering community are honorable and trustworthy persons.

3. The students, faculty members, and administrators of the College of Engineering trust each other to uphold the principles of the Honor Code. They are jointly responsible for precautions against violations of its policies.

4. It is dishonorable for students to receive credit for work that is not the result of their own efforts.

Among other things, the Honor Code forbids plagiarism. To plagiarize is to use another person's ideas, writings, etc. as one's own, without crediting the other person. Thus, you must credit information obtained from other sources, including websites, e-mail or other written communications, conversations, articles, books, etc.

Collaboration

We expect strict adherence to the Engineering Honor Code in all homework assignments. All problem sets (homework assignments) are to be completed on your own. You are encouraged to discuss ideas and techniques broadly with other class members, but all written works, whether in scrap or final form, are to be generated by you working alone unless otherwise expressly stated in the homework assignment. You are not allowed to sit together and work out the details of the problems with anyone. You are not allowed to discuss the problem set with previous class members, nor anyone else who has significant knowledge of the details of the problem set. Nor should you compare your written solutions, whether in scrap paper form or your final work product, to other students (and vice versa). You are also not allowed to possess, look at, use, or in any way derive advantage from the existence of solutions prepared in prior years, whether these solutions were former students' work products or copies of solutions that had been made available by instructors. Violation of this policy is grounds to initiate an action that would be filed with the Dean's office and would come before the College of Engineering's Honor Council. If you find any ambiguity about this policy, it is your responsibility to contact the course staff for clarification.

The final project can be a project with one person or a collaborative project with two people.  More details on the final project will follow.  

Special Accommodations

If you have disabilities or medical conditions that require some form of accommodations, please contact your instructor and the Office of Students with Disabilities at the start of the term so that arrangements can be made to accommodate you.

Notes: The instructor reserves the right to modify course policies and the course calendar according to the progress and needs of the class.