CLEF 2011 Labs

The laboratories are aimed at continuing and expanding on the traditional CLEF track philosophy. Two different forms of labs are offered: benchmarking activities proposing evaluation tasks and comparative performance analyses, and workshop-style labs that explore issues of information access evaluation and related fields.

BENCHMARKING ACTIVITIES

Six benchmarking activities are proposed:

PAN
A benchmarking activity on uncovering Plagiarism, Authorship, and Social Software Misuse.

Organizers:

• Benno Stein,
• Paolo Rosso,
• Moshe Koppel,
• Efstathios Stamatatos,
• Shlomo Argamon,
• Patrick Juola,
• Martin Potthast,
• Alberto Barrón-Cedeño

ImageCLEF Cross Language Image Retrieval Track

A benchmarking activity on the experimental evaluation of image retrieval, focusing on the combination of textual and visual retrieval, including multilingual retrieval and language-independent information access.

Organizers:

• Henning Müller,
• Theodora Tsikrika,
• Stefanie Nowak,
• Alexis Joly,
• Miguel E Ruiz

QA4MRE: Question Answering for Machine Reading Evaluation

A benchmarking activity on the evaluation of Machine Reading systems through Question Answering and Reading Comprehension Tests.
Organizers:

• Anselmo Peñas,
• Eduard Hovy,
• Pamela Forner,
• Alvaro Rodrigo Yuste,
• Richard Sutcliffe

LogCLEF 2011

A benchmarking activity on Multilingual Log File Analysis: Language identification, query classification, success of a query.

Organizers:

• Giorgio Maria Di Nunzio,
  
• Johannes Leveling,
  
• Thomas Mandl

MusiClef

A benchmarking activity on the Evaluation of music search engines that are based both on audio content and on multilingual textual descriptions.

Organizers:

• Nicola Orio,
• David Rizo Valero

CLEF-IP

A benchmarking activity on intellectual property.

Organizers:

• Florina Piroi,
• Mihai Lupu,
• Veronika Zenz,
• Allan Hanbury

WORKSHOP

CHiC 2011 Cultural Heritage in CLEF: From Use Cases to Evaluation in Practice for Multilingual Information Access to Cultural Heritage

The CHiC 2011 workshop aims at moving towards a systematic and large-scale evaluation of cultural heritage digital libraries and information access systems and helping to shape a possible roadmap for it.

Organizers:

• Nicola Ferro,
• Monica Paramita,
• Maria Gäde