Evaluation Exercises for the Semantic Analysis of Text
Organized by ACL-SIGLEX
|
|
There are now many computer programs for automatically determining the sense of a word in context (Word Sense Disambiguation or WSD).
The purpose of Senseval is to evaluate the strengths and weaknesses of such programs with respect to different words, different varieties
of language, and different languages.
Senseval-1 took place in the summer of 1998 for English, French, and Italian,
culminating in a workshop held at Herstmonceux Castle, Sussex, England on September 2-4.
Senseval-2 2took place in the summer of 2001, and was followed by a workshop held in July 2001 in Toulouse, in conjunction with ACL 2001.
Senseval-2 included tasks for Basque, Chinese, Czech, Danish, Dutch, English, Estonian, Italian, Japanese, Korean, Spanish, Swedish.
Senseval-3 took place in March-April 2004, followed by a workshop held in July 2004 in Barcelona, in conjunction with ACL 2004. Senseval-3 included 14 different tasks for core word sense disambiguation, as well as identification of semantic roles, multilingual annotations, logic forms, subcategorization acquisition.
Semeval-2007 took place in July 2007, and was followed by a workshop held in conjunction with ACL in Prague. Semeval-1 included 18 different tasks targeting the evaluation of systems for the semantic analysis of text.
Semeval-2010 took place in July 2010 in Uppsala, Sweden. Check the Semeval-2010 site for more information.
News
- The call for task proposals for Semeval-2 has been issued. Task proposals are due on October 19 2008 (an intent to submit should be emailed to the Semeval-2 organizers by September 21). Check the Semeval-2 site for details.
- All data sets used during the Senseval-3 evaluations are now in the public domain. The Senseval-3 proceedings and panel minutes are also available. Check the Senseval-3 website.
|
|
|
