PA5 — The Interpreter

Programming assignments 2 through 4 involved the constructed of the front-end (lexer, parser) and gatekeeping (semantic analyzer) stages of an interpreter. In this assignment you will write the code that performs the execution and interpretation of valid programs.

You may complete this assignment in OCaml, Haskell, JavaScript, Python or Ruby.

You may work in a team of two people for this assignment. You do not need to keep the same teammate. The course staff are not responsible for finding you a willing teammate.


For this assignment you will write an interpreter. Among other things, this involves implementing the operational semantics specification of Cool. You will track enough information to generate legitimate run-time errors (e.g., dispatch on void). You do not have to worry about "malformed input" because the semantic analyzer (from PA4) has already ruled out bad programs.

You will also write additional code to unserialize the class and implementation maps produced by the semantic analyzer and the parse tree produced by the parser.

The Specification

You must create three artifacts:

  1. A program that takes a single command-line argument (e.g., That argument will be an ASCII text Cool class map, implementation map, and AST file (as described in PA4). Your program must execute (i.e., interpret) the Cool program described by that input. If your program is called interp, invoking interp should yield the same output as cool Your program will consist of a number of OCaml files, a number of Python files, or a number of Ruby files.
    • You will only be given .cl-type files from programs that pass the semantic analysis phase of the reference interpreter. You are not responsible for correctly handling (1+"hello") programs.
  2. A plain ASCII text file called readme.txt describing your design decisions and choice of test cases. See the grading rubric. A few paragraphs should suffice.
  3. Testcases,, and The testcases should exercise interpreter and run-time error corner cases.

Error Reporting

To report an error, write the string

ERROR: line_number: Exception: message

to standard output and terminate the program. You may write whatever you want in the message, but it should be fairly indicative. Example erroneous input:

class Main inherits IO {
  my_void_io : IO ; -- no initializer => void value
  main() : Object {
    my_void_io.out_string("Hello, world.\n")
  } ;
} ;

Corresponding example error report output:

ERROR: 4: Exception: dispatch on void


You will have to handle all of the internal functions (e.g., IO.out_string) that you first encountered in PA4.

You can do basic testing as follows:

$ cool --type
$ cool >& reference-output
$ my-interp >& my-output
$ diff my-output reference-output

Note that this time, whitespace and newlines matter for normal output. This is because you are specifically being asked to implement IO and substring functions.

You should implement all of the operational semantics rules in the Reference Manual. You will also have to implement all of the built-in functions on the five Basic Classes.

PA5t — Creating PA5 Tests

PA5t is a preliminary testing exercise that introduces a form of test-driven development or mutation testing into our software development process and requires you to construct a high-quality test suite.

The goal of PA5t is to leave you with a high-quality test suite of Cool programs that you can use to evaluate your own PA5 Interpreter. Writing an interpreter requires you to consider many corner cases (perhaps even more than in PA4!) when reading the formal operational semantics rules in the Cool Reference Manual. While you you can check for correct "positive" behavior by comparing your interpreter's output to the reference interpreters's output on the usual "good" Cool programs, it is comparatively harder to check for "corner case" behavior.

If you fail to construct a rich test suite of semantically-valid tricky programs you will face a frustrating series of "you fail held-out negative test x" reports for PA5 proper, which can turn into unproductive guessing games. Because students often report that this is frustrating (even though it is, shall we say, infinitely more realistic than making all of the post-deployment tests visible in advance), the PA5t preliminary testing exercise provides a structured means to help you get started with the constuction of a rich test suite.

The course staff have produced 22 variants of the reference compiler, each with a secret intentionally-introduced defect related to Interpretation. A high-quality test suite is one that reveals each introduced defect by showing a difference between the behavior of the true reference compiler and the corresponding buggy verison. You desire a high-quality test suite to help you gain confidence in your own PA5 submission.

For PA5t, you must produce syntactically valid Cool programs (test cases). There are 22 separate held-out seeded interpreter bugs waiting on the grading server. For each bug, if one of your tests causes the reference and the buggy version to produce difference output (that is different stdout/stderr), you win: that test has revealed that bug. For full credit your tests must reveal at least 17 of the 22 unknown defects.

The secret defects that we have injected into the reference compiler correspond to common defects made by students in PA5. Thus, if you make a rich test suite for PA5t that reveals many defects, you can use it on your own PA5 submission to reveal and fix your own bugs!

Video Guides

A number of Video Guides are provided to help you get started on this assignment on your own. The Video Guides are walkthroughs in which the instructor manually completes and narrates, in real time, the first part of this assignment — including a submission to the grading server. They include coding, testing and debugging elements.

If you are still stuck, you can post on the forum, approach the TAs, or approach the professor. The use of online instructional content outside of class weakly approximates a flipped classroom model. Click on a video guide to begin, at which point you can watch it fullscreen or via Youtube if desired.

PA5t — Test Cases
PA5c — Interpreting

PA5 — Automating Testing and Minimizing Tests (Delta Debugging)

What to Turn In For PA5t

For PA5t you should turn in (electronically):

Your zip file may also contain:

Hint: All of the usual tactics apply (from randomly-generating programs to permuting one symbol in each operational semantics rule to reading the prose descriptions in the CRM and looking for words like "must" to digging through the reference compiler binary).

PA5c — Checkpoint

PA5c is a checkpoint for PA5. The Interpreter is a large and complicated assignment; we do not want you to fall behind.

For the PA5c checkpoint you will only be tested on something akin to If you can interpret that, you pass the checkpoint. (You can "cheat" the checkpoint by hard-coding output for that single test case, but you're ultimately only hurting yourself!) The goal of the checkpoint is not to do the minimal amount of work possible for this program, but instead to do the greatest amount possible now so that you have plenty of time for the rest of the features later.

You must turn in a zip file containing these files:

  1. source_files — your implementation, including
    • main.rb or
    • or
    • main.js or
    • main.hs or

Your zip file may also contain:

What to Turn In For PA5

You must turn in a zip file containing these files:

  1. readme.txt — your README file
  2. — a novel testcase
  3. — a novel testcase
  4. — a novel testcase
  5. — a novel testcase
  6. source_files — your implementation, including
    • main.rb or
    • or
    • main.js or
    • main.hs or

Your zip file may also contain:

Language-Specific Hints

Python hint:

I was performing the computation 4 / -59. In a very strange turn of events, this computation returns 0 in a simple C program (and, apparently, in OCaml), but -1 in Python. Bwah?

Python's division truncation works differently from C's. This is more than a little annoying, and certainly a "gotcha".

previous student Charles Eckman

Charles reports that he was able to solve the problem by adding from __future__ import division to the top of his Python source, which makes the division operator always return a floating-point result. That result can then be cast to an integer via the int() constructor.

Grading Rubric

PA5 Grading (out of 100 points):