{
  "version": "https://jsonfeed.org/version/1.1",
  "title": "Cyrus Omar's feed",
  "home_page_url": "https://web.eecs.umich.edu/~comar",
  "feed_url": "https://web.eecs.umich.edu/~comar/feed.json",
  "icon": "https://web.eecs.umich.edu/~comar/assets/favicon.ico",
  "authors": [
    {
      "name": "Cyrus Omar",
      "url": "https://web.eecs.umich.edu/~comar/",
      "avatar": "https://web.eecs.umich.edu/~comar/images/anil-headshot.webp"
    }
  ],
  "language": "en-US",
  "items": [
    {
      "id": "https://web.eecs.umich.edu/~comar/notes/lively-typed-tables",
      "content_html": "<p>Most data analysis happens in dynamic languages like <strong>Python</strong>, <strong>R</strong>, <strong>MATLAB</strong>, and <strong>Julia</strong>\nusing computational notebook interfaces like <strong>Jupyter</strong> and <strong>R Studio</strong>.</p>\n<p>These are flexible systems, but dynamic checking leaves critical information about\n<em>table schemas</em>, i.e. a table's column names and corresponding types,\ncompletely latent in the operations that are performed.</p>\n<p>Knowing the schema of a table generated during a calculation is useful for\nseveral reasons. It allows your editor (even when used by an AI) to provide more accurate autocomplete and\nerror localization, often before you waste your time running an expensive\nbut incorrect computation.</p>\n<p>Making schemas explicit, e.g. at function and module boundaries, is also critical to\ninteroperability and reuse, key concerns as scientific\nwork scales up to planetary scale (see our <a href=\"/papers/2025-fair-commons-propl\">FAIR case for a live computational commons</a>).</p>\n<p>Shifting to a statically typed approach could help matters, with table schemas tracked as types.\nHowever, there is <strong>a major problem with statically typed systems for working with tables</strong>: they struggle with\n<strong>data-dependent operations</strong> where the schema of a function's output depends on the values in its input.</p>\n<p>For example,\nunstacking a long table into a wide one like in the example below produces a table whose column names and types\ndepend\non the values (here, in the &quot;element&quot; column) of the input! No ordinary static type checker can encode the type of this kind of operation\nprecisely.</p>\n<p><img src=\"/images/notes/lively-typed-tables-unstack.png\" alt=\"%c\" title=\"Unstacking a table of weather observations. The new columns come from the data, so there is no tmean column; live typing flags the access.\" ></p>\n<p>In <a href=\"/papers/2026-typed-tables-oopsla\">Interactive Data Analysis with Lively Typed Tables</a>,\nwhich <a href=\"https://bandukwala.me/\">Alexander Bandukwala</a> presented at OOPSLA 2026, we take a different\napproach in <a href=\"/projects/hazel\">Hazel</a>:</p>\n<ul>\n<li><strong>Tables are semantically just lists of labeled tuples</strong>, with a few gradually typed\noperations (where the type is left statically unknown) for working with columns as strings.</li>\n<li><strong>Live typing</strong> feeds the types Hazel observes while running your program back\ninto the static type checker. Because Hazel is live and keeps running programs even when they\nhave holes and errors, it can report that a column produced by an unstack\ndoesn't exist right where you access it, before you edit (or evaluate) further (e.g. in the example above, there is an error on the incorrect projection of <code>tmean</code>).</li>\n</ul>\n<p>This is a straightforward approach, but it works beautifully!</p>\n<p>Going further, we build a number of <em>direct manipulation editing</em> tools for working with tables directly into Hazel as well, so you can do exploratory data analysis directly in Hazel and, unlike in tools like spreadsheets, encode the manipulations you performed as reusable and modifiable code.</p>\n<p><img src=\"/images/notes/lively-typed-tables-rich-probe.png\" alt=\"%c\" title=\"Rich table probes. (a) The column header menu for Year suggests type conversions based on the live type of the table. (b) Each choice is written back into the program as code.\" ></p>\n<p>The paper evaluates the overall Hazel design against the <a href=\"https://github.com/brownplt/B2T2\">Brown Benchmark for Table Types</a>,\nwhere Hazel expresses every example, and in a lab study with seven\nparticipants doing data cleaning and analysis. People liked it!</p>\n<p>You can try tables in Hazel\ntoday (see the tables tutorial from the dropdown at the top, and the Tables page in the documentation at\n<a href=\"https://hazel.org/build/dev/\">hazel.org/build/dev</a>). There is a\n<a href=\"https://www.youtube.com/watch?v=aCbrzAnBEbM\">recording of the talk (14m)</a> where Alexander gives several nice live demos of these ideas.</p>\n<p><strong>During my sabbatical in Cambridge</strong>, I hope to try porting more sophisticated data analyses from folks working in climate modeling of all sorts\ninto this system and continue scaling it up until the point where Hazel really is a better tool overall for this kind of work than existing\nsystems. We've got a ways to go with performance but based on our lab study, we think we're quite close on the UI and semantics front!</p>\n",
      "url": "https://web.eecs.umich.edu/~comar/notes/lively-typed-tables",
      "title": "Lively typed tables in Hazel",
      "summary": "Alexander Bandukwala's OOPSLA 2026 paper brings static types to exploratory data analysis without giving up the flexibility of data frames.",
      "date_published": "2026-10-31T00:00:00.000000Z",
      "date_modified": "2026-10-31T00:00:00.000000Z",
      "authors": [
        {
          "name": "Cyrus Omar",
          "url": "https://web.eecs.umich.edu/~comar/",
          "avatar": "https://web.eecs.umich.edu/~comar/images/anil-headshot.webp"
        }
      ],
      "tags": [
        "hazel",
        "oopsla"
      ],
      "attachments": [],
      "_references": [
        {
          "url": "https://doi.org/10.1145/3759536.3763802",
          "doi": "10.1145/3759536.3763802",
          "cito": [
            "citesAsSourceDocument"
          ]
        },
        {
          "url": "https://doi.org/10.1145/3839501",
          "doi": "10.1145/3839501",
          "cito": [
            "citesAsSourceDocument"
          ]
        }
      ]
    }
  ]
}