• CHiC 2011: Cultural Heritage in CLEF

    Information systems that access Cultural Heritage (CH) deal with effective information access to cultural heritage material held in large-scale digital libraries containing data from libraries, archives, museums, and audio-visual archives. However, access to these objects still poses several challenges related to the heterogeneous media types (texts, audio or video files) and user groups (novice and expert users), often with specialized information needs. Objects are provided by meta data, usually in their national language and with specified technical vocabularies suited for their particular domains. Previous research has focused on the satisfaction of user information needs by retrieving relevant “cultural assets” irrespective of the media type, location or language in which information objects are expressed. Despite the growth of digital libraries and the research carried out in this field, much less is done to establish standard evaluation criteria and methods.

    The CHiC2011 – Cultural Heritage in CLEF: From Use Cases to Evaluation in Practice for Multilingual Information Access to Cultural Heritage workshop at CLEF 2011 will start to investigate these issues by surveying evaluation efforts in the cultural heritage field as well as defining user scenarios and identifying possible relevant metrics. The workshop provides an overview of previous or current evaluation activities and seeks to introduce an exchange about future efforts that needs to be addressed in the CH field.

    Invited talks will address use cases, evaluation approaches, and best practices from an institutional point of view as well as the experiences from large-scale evaluation campaigns such as CLEF, TREC or INEX . Participants are encouraged to include short statements and participate in the discussions concerning complementary efforts, projects, initiatives and available test data. Based on the speakers input and group discussions the second part of the workshop aims to identify possible synergies between evaluation frameworks within CH projects and campaigns as well as the development of use cases and usage scenarios that can be applied to CH information systems and evaluation tasks.

    10:30-10:45 Introduction
    10:45-12:30 Keynote Talks:
    User-centric approaches to annotating and presenting the audiovisual memory of the Netherlands, Johan Oomen
    Evaluation of Cultural Heritage Digital Collections: The DiLEO Perspective, Christos Papatheodorou
    Searching Digital Heritage: Putting IR Evaluation in Context, Jaap Kamps
    12:30-14:00 Lunch
    14:00-14:20 Overview: Interaction Patterns and Evaluation Efforts in the Cultural Heritage Domain
    14:20-15:00 Participants Statements
    15:00-16:00 Interactive Discussion: Roadmap for Alternative Evaluation & Proposal of a First Evaluation Task in the Cultural Heritage Domain



The registration desk will be open throughout the conference, on the first floor of Casa 400. Additional opening hours are:
- Sunday 18 September, from 17.00 to 19.00
- Monday 19 September, from 08.00 onwards

Fee waivers

CLEF 2011 receives last-minute sponsorship for student registration fee waivers. Apply (with motivation + proof of stud. status) at clef2011[at]

Schedule Change

The notifications of acceptance of CLEF 2011 conference paper is postponed to Monday 13

CLEF 2011 Registrations

The CLEF 2011 registrations are now open.


General chairs

Maarten de Rijke
Julio Gonzalo

Program chairs

Jaana Kekäläinen
Mounia Lalmas

Lab chairs

Vivien Petras
Paul Clough

Organization chair

Emanuele Pianta

Resource and Publicity chair

Khalid Choukri

Technical management

Pamela Forner
Giovanni Moretti

Local organizers

Richard Berendsen
Bouke Huurnink
Edgar Meij
Wouter Weerkamp