SENSEVAL.ORG


Mission

Senseval is the international organization devoted to the evaluation of Word Sense Disambiguation Systems.

Its mission is to organise and run evaluation and related activities to test the strengths and weaknesses of WSD systems with respect to different words, different aspects of language, and different languages. It's underlying goal is to further our understanding of lexical semantics and polysemy.

Senseval is run by small committee under the auspices of ACL-SIGLEX (the Special Interest Group on the LEXicon of the Association for Computational Linguistics).

Read here the Senseval Constitution.

Word Sense Disambiguation

Word sense disambiguation (WSD) is the problem of deciding which sense a word has in any given context. It has been very difficult to formalize the process of disambiguation, which humans can do so effortlessly. For virtually all applications of language technology, word sense ambiguity is a potential source of error. Take machine translation (MT). If the English word 'drug' translates into French as either 'drogue' or 'médicament', then an English-French MT system needs to disambiguate every use of 'drug' in order to make the correct translations. Similarly, information retrieval systems may erroneously retrieve documents about an illegal narcotic when the item of interest is a medication; analogously, information extraction systems may make wrong assertions; and, text-to-speech will confuse violin bows for ship's bows.

In actual applications, WSD is often fully integrated into the system and often cannot be separated out (for instance, in information retrieval WSD is often not done explicitly but is just by-product of query to document matching). But in order to study and evaluate WSD, Senseval has, to date, concentrated on standalone, generic systems for WSD.

Evaluation activities

The success of any project in WSD is clearly tied to the evaluation of WSD systems. SENSEVAL was started in 1997, following a workshop, Tagging with Lexical Semantics: Why, What, and How?, held at the conference on Applied Natural Language Processing.

Senseval-1 was held in the summer of 1998, culminating in a workshop at Herstmonceux Castle, England on September 2-4 (Kilgarriff and Palmer 2000). Following the success of the first workshop, Senseval-2, supported by EURALEX, ELSNET, EPSRC, and ELRA, was organized in 2000-2001. The Second International Workshop on Evaluating Word Sense Disambiguation Systems was held in conjunction with ACL-2001 on July 5-6, 2001 in Toulouse.


Advising committee

Nicoletta Calzolari, Istituto di Linguistica Computazionale - CNR, Pisa
Phil Edmonds Sharp Laboratories of Europe
Adam Kilgarriff, ITRI, University of Brighton
Rada Mihalcea University of North Texas
Martha Palmer, University of Pennsylvania
David Yarowsky, John Hopkins University

Organization committee

Richard Wicentowski (co-chair), Swarthmore College, USA
Lluis Marquez (co-chair), Universitat Politecnica de Catalunya, Catalonia
Eneko Agirre (co-chair), Basque Country University, Basque Country
Dimitrios Kokkinakis (secretary), Goteborg University
Ergin Altintas, Turkish Naval Academy, Turkey
Veronique Hoste, University of Antwerp
Nancy Ide, Vassar College, USA
Hwee Tou Ng, National University of Singapore
Amruta Purandare, University of Pittsburgh, USA
German Rigau, Basque Country University, Basque Country
Carlo Strapparava, ITC-IRST, Italy
Deniz Yuret, Koc University, Turkey

Past committees

2001-2004
Phil Edmonds (co-chair), Sharp Laboratories of Europe
Rada Mihalcea (co-chair), University of North Texas
Dimitrios Kokkinakis (secretary), Goteborg University
Sadao Kurohashi, The University of Kyoto
Bernardo Magnini, IRST, Italy
Diana McCarthy, University of Sussex
Hwee Tou Ng, National University of Singapore
Ted Pedersen, University of Minnesota, Duluth
Judita Preiss, University of Cambridge
German Rigau, Universitat Politecnica de Catalunya