You are here

Home

ImageCLEF 2023

Sponsors

Motivation

Following the ImageCLEF 2003-2022 evaluation campaigns, ImageCLEF 2023 (https://www.imageclef.org/, https://www.facebook.com/imageclef, https://twitter.com/imageclef), the Multimodal Challenge organized as part of the CLEF Initiative Labs, continues the tradition and is set to promote the evaluation of technologies for #annotation, #indexing, #classification and #retrieval of multimodal data, with the objective of providing information access to large collections in various usage scenarios and domains.

The campaign offers several research tasks that welcome participation from teams around the world. ImageCLEF 2023 will focus again on a diversity of applications, namely medical, social media and Internet, and content recommendation, allowing to exploit synergies between different tasks. Like 2022, ImageCLEF 2023 will also lay the foundations of new innovative tasks, while some of the older tasks are discontinued.

The results of the campaign appear in the working notes proceedings, published by CEUR Workshop Proceedings (ceur-ws.org, e.g., 2022 http://ceur-ws.org/Vol-3180/). Selected contributions among the participants, will be invited for publication in the following year in the Springer Lecture Notes in Computer Science (LNCS) together with the annual lab overviews (e.g., 2022 https://link.springer.com/book/10.1007/978-3-031-13643-6).

Target communities involve (but are not limited to): information retrieval (e.g., text, vision, audio, multimedia, social media, sensor data), machine learning, deep learning, data mining, natural language processing, image and video processing; with special emphasis on the challenges of multi-modality, multi-linguality, and interactive search.

Stay tuned with us for the latest information and updates by joining us on the ImageCLEF social media accounts: Twitter #imageclef, and Facebook @ImageClef.

ImageCLEF2023 schedule

Each of the tasks sets its own schedule, so please check the corresponding task webpage for specific dates. A (tentative) global schedule can be found below:

  • 14.11.2022: registration opens for all ImageCLEF tasks
  • 17.01.2023: development data release starts (depends on the task)
  • 14.03.2023: test data release starts (depends on the task)
  • 10.05.2023: deadline for submitting the participants runs (depends on the task)
  • 17.05.2023: release of the processed results by the task organizers (depends on the task)
  • 05.06.2023: deadline for submission of working notes papers by the participants
  • 23.06.2023: notification of acceptance of the working notes papers
  • 07.07.2023: camera ready working notes papers
  • 18-21.09.2023: CLEF 2023, Thessaloniki, Greece

The CLEF Conference

ImageCLEF lab and all its tasks are part of the Conference and Labs of the Evaluation Forum: CLEF 2023. CLEF 2023 consists of an independent peer-reviewed workshops on a broad range of challenges in the fields of multilingual and multimodal information access evaluation, and a set of benchmarking activities carried in various labs designed to test different aspects of mono and cross-language information retrieval systems. More details about the conference can be found here. Also there is more information about the CLEF Initiative.

Programme of ImageCLEF at the CLEF 2023 Conference

Please check the programme here:

Participant registration

Important: We do not longer accept registrations to ImageCLEF2023, but we are looking forward to welcome you to ImageCLEF2024.

Registration Form

A team can be registered by a representative member by filling in the ImageCLEF Registration Form (for any information, please contact Ivan Eggel ivan.eggel@hevs.ch or Ana-Maria Drăgulinescu ana.dragulinescu@upb.ro.)

EUA uploading

Team’s representative member has to fill in and submit one EUA for each ImageCLEF task they want to participate in (please note that some tasks do not require an EUA). The Registration Form has special fields for documents upload and are made available once the representative member selects to register for a specific task.

An ImageCLEF participant is considered as registered for a task as soon as he/she has uploaded a valid EUA that was approved by the ImageCLEF task (examples of how the EUAs should be filled in can be found here: Caption 2019 EUA example, Tuberculosis 2019 EUA example and VQA 2019 EUA example).

After validating your EUA, the task will grant read access to the dataset via the team representative’s Github account. If the registration needs to be modified, the team’s representative member will receive an e-mail notification.

The EUAs templates are available here:
Caption 2023 EUA
GANs 2023 EUA
VQA 2023 EUA
Fusion 2023 EUA
Aware 2023 EUA

Form update

Each representative will be able to modify the registration, except the already uploaded documents. In case it is necessary to resubmit new versions of EUAs, you will receive an e-mail notification from the registered tasks. Thus, the team’s representative is allowed to:

  • change the representative member’s and team’s information using the same form just by sending another response via the Google Form. The team’s representative member and the organizers of all the tasks to which one registered will be notified.
  • register to new tasks until the task registration deadline is over, using the same form just by sending another response via the Google Form.

Important notes:

  • Please DO NOT register on the CLEF websites for the ImageCLEF tasks. Our registrations will be automatically transmitted to CLEF.
  • The representative members are not allowed to register twice.

The Tasks

ImageCLEF 2023 proposes 4 main tasks:

  • (5th edition) ImageCLEFmedical: Medical information can be used in a variety of scenarios and this task will combine the most popular medical task of ImageCLEF with three new tasks and continue the idea of 2019 by mixing various applications, namely:
  • (3rd edition) ImageCLEFaware: The images available on social networks can be exploited in ways users are unaware of when initially shared, including situations that have serious consequences for the users’ real lives. The task addresses the development of algorithms which raise the users’ awareness about real-life impact of online image sharing.
  • (2nd edition) ImageCLEFfusion: Despite the current advances in knowledge discovery, single learners do not produce satisfactory performances when dealing with complex data, such as class imbalance, high-dimensionality, concept drift, noisy data, multimodal data, subjective annotations, etc. This task aims to fill this gap by exploiting novel and innovative late fusion techniques for producing a powerful learner based on the expertise of a pool of classifiers.
  • (new) ImageCLEFrecommending: This task focuses on content-recommendation for cultural heritage content in 15 broad themes that have been curated by experts in the Europeana Platform (https://www.europeana.eu/en/themes). Despite current advances, there is limited understanding how well these perform and how relevant they are for the final end users. This task aims to fill this gap by developing ground truth data of recommendations in the above themes and by allowing benchmarking different recommendation systems and methods.

Overview Paper

When referring to ImageCLEF 2023, please cite the following publication:

  • Bogdan Ionescu, Henning Müller, Ana-Maria Drăgulinescu, Wen-wai Yim, Asma Ben Abacha, Neal Snider, Griffin Adams, Meliha Yetisgen, Johannes Rückert, Alba García Seco de Herrera, Christoph M. Friedrich, Louise Bloch, Raphael Brüngel, Ahmad Idrissi-Yaghir, Henning Schäfer, Steven A. Hicks, Michael A. Riegler, Vajira Thambawita, Andrea Storås, Pål Halvorsen, Nikolaos Papachrysos, Johanna Schöler, Debesh Jha, Alexandra-Georgiana Andrei, Ahmedkhan Radzhabov, Ioan Coman, Vassili Kovalev, Alexandru Stan, George Ioannidis, Hugo Manguinhas, Liviu-Daniel Ștefan, Mihai Gabriel Constantin, Mihai Dogariu, Jérôme Deshayes, Adrian Popescu, Overview of the ImageCLEF 2023: Multimedia Retrieval in Medical, Social Media and Recommender Systems Applications, in Experimental IR Meets Multilinguality, Multimodality, and Interaction.Proceedings of the 14th International Conference of the CLEF Association (CLEF 2023), Springer Lecture Notes in Computer Science LNCS, Thessaloniki, Greece, September 18-21, 2023.
  • BibTex:
    @inproceedings{ImageCLEF2023,

    author = {Bogdan Ionescu and Henning M\"uller and Ana{-}Maria Dr\u{a}gulinescu and Wen{-}wai Yim and Asma {Ben Abacha} and Neal Snider and Griffin Adams and Meliha Yetisgen and Johannes R\"uckert and Alba {Garc\’{\i}a Seco de Herrera} and Christoph M. Friedrich and Louise Bloch and Raphael Br\"ungel and Ahmad Idrissi{-}Yaghir and Henning Sch\"afer and Steven A. Hicks and Michael A. Riegler and Vajira Thambawita and Andrea Storås and Pål Halvorsen and Nikolaos Papachrysos, Johanna Schöler, Debesh Jha, Alexandra{-}Georgiana Andrei, Ahmedkhan Radzhabov, Ioan Coman, Vassili Kovalev, Alexandru Stan, George Ioannidis and Hugo Manguinhas and Liviu{-}Daniel \c{S}tefan and Mihai Gabriel Constantin and Mihai Dogariu and J\'er\^ome Deshayes and Adrian Popescu}
    title = {{Overview of ImageCLEF 2023}: Multimedia Retrieval in Medical, SocialMedia and Recommender Systems Applications},
    booktitle = {Experimental IR Meets Multilinguality, Multimodality, and Interaction},
    series = {Proceedings of the 14th International Conference of the CLEF Association (CLEF 2023)},
    year = {2023},
    publisher = {Springer Lecture Notes in Computer Science LNCS},
    pages = {},
    month = {September 18-21},
    address = {Thessaloniki, Greece}
    }

The Organising Committee

Overall coordination

  • Bogdan Ionescu <bogdan.ionescu(at)upb.ro>, Politehnica University of Bucharest, Romania
  • Henning Müller <henning.mueller(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland
  • Ana-Maria Drăgulinescu <ana.dragulinescu(at)upb.ro>, Politehnica University of Bucharest, Romania

Technical support

  • Ivan Eggel <ivan.eggel(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland

ImageCLEF - Cross-language image retrieval evaluations