ImageCLEF 2024

Sponsors

Motivation

Given the successful ImageCLEF 2003-2023 benchmarking campaigns, ImageCLEF 2024 edition (https://www.imageclef.org/, https://www.facebook.com/imageclef, https://twitter.com/imageclef) will continue to evaluate the technologies for annotation, indexing, classification and retrieval of multimodal data, with its main objective residing in providing access to large collections of multimodal data for multiple usage scenarios and domains.
Considering the experience of the last editions, ImageCLEF 2024 will continue approaching a diversity of applications, namely medical, social media and Internet, and recommending, giving to the participants the opportunity to deal with interdisciplinary approaches and domains. ImageCLEF 2024 will continue the experience of the engaging tasks from 2023 edition, while some of the other tasks are discontinued.
ImageCLEF 2024 is organized as part of the CLEF Initiative Labs.

The results of the campaign appear in the working notes proceedings, published by CEUR Workshop Proceedings (ceur-ws.org, e.g., 2023 http://ceur-ws.org/Vol-3497/). Selected contributions among the participants, will be invited for publication in the following year in the Springer Lecture Notes in Computer Science (LNCS) together with the annual lab overviews (e.g., 2023 https://link.springer.com/book/10.1007/978-3-031-42448-9).

The target audience for ImageCLEF 2024 is mainly expected to be from the multimodal data annotation and retrieval community, from fields such as computer vision, image information retrieval and digital image processing. Due to the success and the specific nature of the medical tasks, a significant part of the audience will come from the medical informatics, machine learning and pattern recognition community.

Stay tuned with us for the latest information and updates by joining us on the ImageCLEF social media accounts: Twitter #imageclef, and Facebook @ImageClef.

ImageCLEF2024 schedule

Each of the tasks sets its own schedule, so please check the corresponding task webpage for specific dates. A (tentative) global schedule can be found below:

• 30.11.2023: registration opens for all ImageCLEF tasks
• 22.04.2024: registration closes for all ImageCLEF tasks
• 17.01.2024: development data release starts (depends on the task)
• 14.03.2024: test data release starts (depends on the task)
• 06.05.2024 : deadline for submitting the participants runs (depends on the task)
• 13.05.2024 : release of the processed results by the task organizers (depends on the task)
• 31.05.2024 : deadline for submission of working notes papers by the participants
• 21.06.2024: notification of acceptance of the working notes papers
• 08.07.2024 : camera ready working notes papers
• 09-12.09.2024: CLEF 2024, Grenoble, France

The CLEF Conference

ImageCLEF lab and all its tasks are part of the Conference and Labs of the Evaluation Forum: CLEF 2024. CLEF 2024 will be hosted by the University of Grenoble Alpes, France, 9-12 September 2024 and consists of an independent peer-reviewed workshops on a broad range of challenges in the fields of multilingual and multimodal information access evaluation, and a set of benchmarking activities carried in various labs designed to test different aspects of mono and cross-language information retrieval systems. More details about the conference can be found here. Also there is more information about the CLEF Initiative.

Programme of ImageCLEF at the CLEF 2024 Conference

“Overview of the ImageCLEF 2024 Lab”
Bogdan Ionescu, National University of Science and Technology Politehnica Bucharest, Romania
Henning Müller, University of Applied Sciences Western Switzerland, Sierre, Switzerland
Alexandra Andrei, National University of Science and Technology Politehnica Bucharest, Romania

1. “Overview of ImageCLEFMedical 2024 - Caption Task”, Hendrik Damm, University of Applied Arts and Science Dortmund - physical
2. “Overview of ImageCLEFMedical 2024 - VQA”, Steven Hicks, SimolaMet
3. “Overview of the MEDIQA-MAGIC Task: Multimodal And Generative TelemedICine in Dermatology”, Asma Ben Abacha, Microsoft, USA - online
4. "Overview of the 2024 ImageCLEFmedical GANs Task - Investigating Generative Models' Impact on Biomedical Synthetic Images" - Alexandra-Georgiana Andrei, National University of Science and Technology - physical

---

5. “Overview of Image Retrieval/Generation for Arguments”, Johannes Kiesel, Bauhaus-Universität Weimar - physical
6. “Overview of ImageCLEF - ToPicto 2024 tasks” - Cécile Macaire, Didier Schwab, Benjamin Lecouteux, Univ. Grenoble Alpes, CNRS, Grenoble INP*, LIG, 38000 Grenoble, France - physical

1. “Text-to-Picto using Lexical simplification”, Ananth Narayanan P, Abbhinav Elliah, Bhuvan S, P Mirunalini, Sri Sivasubramaniya Nadar College of Engineering, Tamil Nadu, India
2. Questions / Feedback
3. Conclusion and Perspectives - Cécile Macaire, Didier Schwab, Benjamin Lecouteux, Université Grenoble Alpes

1. “VisionQAries at MEDIQA-MAGIC 2024: Small Vision Language Models for Dermatological Diagnosis”, Maciej Morawski, Patrycja Cieplicka, Jolia Kłos, Independent Researchers, Warsaw, Poland - online

1. “MMCP Team at ImageCLEFmed MEDVQA-GI 2024 Task: Diffusion Models for Text-to-Image Generation of Colonoscopy Images”, Elena Tutubalina, HSE University, Kazan Federal University, Artificial Intelligence Research Institute, Russia - physical
2. “Advancing AI-Powered Medical Image Synthesis: Insights from MedVQA-GI Challenge Using CLIP, Fine-Tuned Stable Diffusion, and Dream-Booth + LoRA”, Ojonugwa Ejiga Peter, Morgan State University, USA - online

1. "AUEB NLP Group at ImageCLEFmedical Caption 2024" - Anna Chatzipapadopoolou, MSc student, Athens University of Economics and Business, Georgios Moschovis, PhD student and research fellow, Athens University of Economics and Business and Archimedes Unit/Athena RC
2. “Predicting Captions and Detecting Concepts for Medical Images: Contributions of the DBS-HHU Team to ImageCLEFmedical Caption 2024” - Heiko Kauschke, Heinrich-Heine-Universität Düsseldorf, Germany

1. "AI Multimedia Lab at ImageCLEFmedical GANs 2024: Deep Learning Approaches for Analyzing Synthetic Medical Images" - Alexandra-Georgiana Andrei, National University of Science and Technology Politehnica Bucharest, Romania - physical
2. "Analyzing Generated Images using Machine Learning" -Diana Miranda, Department of Information Technology, Goa College of Engineering, Farmagudi, Ponda, Goa, India, - online
3. "Fingerprint Identification of Generative Models Using a MultiFormer Ensemble Approach" -Md. Ismail Siddiqi Emon, Department of Computer Science, Morgan State University, Baltimore, USA - online

1. "AUEB NLP Group at ImageCLEFmedical Caption 2024" - Anna Chatzipapadopoolou, MSc student, Athens University of Economics and Business, Georgios Moschovis, PhD student and research fellow, Athens University of Economics and Business and Archimedes Unit/Athena RC
2. “Predicting Captions and Detecting Concepts for Medical Images: Contributions of the DBS-HHU Team to ImageCLEFmedical Caption 2024” - Heiko Kauschke, Kirill Bogomasov, and Stefan Conrad

1. "ImageCLEFmedical 2024 Caption: Concept Detection and Caption Prediction", Hendrik Damm, Henning Schäfer and Christoph M. Friedrich, University of Applied Sciences and Arts Dortmund, Germany

Participant registration

ImageCLEF is an evaluation lab and starting from this edition it will be managed by the ImageCLEF team through the AI4Media benchmarking platform (based on Codalab). The main features of the platform are: online registration system, end-user agreement submission, data distribution, a leaderboard system as well as interaction with participants, reducing the administrative overhead. The platform is now up for participants' registration.

https://ai4media-bench.aimultimedialab.ro/competitions/public/

Please note that the registration is a two-step process. In order to participate in an ImageCLEF task, you must first accept its specific terms and conditions, which are listed on the 'Get Started -> Terms' tab of the tasks hosted on the AI4MediaBench platform. The registration form is mandatory. Within the form, you have the option to choose one or more ImageCLEF tasks from the provided list. Once the form is completed, your application will be applicable to all selected tasks hosted on AI4MediaBench. After completing the form, you must apply to the tasks you have selected in the terms and conditions via the 'Participate' tab of the tasks.

Please do not register separately on the official CLEF 2024 registration page. Registrations on the AI4Media Bench Platform will automatically be transmitted to CLEF.
In case the sub(task) is not present on the AI4Media Bench Platform, please contact the task responsibles (listed on the bottom of each task page).

The Tasks

ImageCLEF 2024 proposes 4 main tasks:

• (8th edition) ImageCLEFmedical: Multimodal data can be used in different scenarios. For example, manual generation of the knowledge of medical images is a time-consuming process prone to human error. As this process requires assistance for the better and easier diagnoses of diseases that are susceptible to radiology screening, it is important that we better understand and refine automatic systems that aid in the broad task of radiology-image metadata generation. Thus, we proposed:
  • (8th edition) automatic image captioning;
  • (2nd edition) synthetic medical images created via GANs;
  • (2nd edition) Visual Question Answering for colonoscopy images;
  • (2nd edition) Multimodal And Generative TelemedICine (MAGIC).
• (3rd edition) ImageCLEF Retrieval for Arguments: Good images provide powerful visual communication and can strengthen an argumentative position. However, despite recent advances, conventional image retrieval is not well suited to finding images that support a particular point of view. The task is to fill this gap and develop techniques for finding suitable images for argumentative texts.
• (2nd edition) ImageCLEFrecommending: This task focuses on content-recommendation for cultural heritage content in 15 broad themes that have been curated by experts in the Europeana Platform (https://www.europeana.eu/en/themes). Despite current advances, there is limited understanding how well these perform and how relevant they are for the final end users. This task aims to fill this gap by developing ground truth data of recommendations in the above themes and by allowing benchmarking different recommendation systems and methods.
• (1st edition) ImageCLEFtoPicto: This task introduces two new challenges whose objective is to provide a translation in pictograms from a natural language, either from (i) text or (ii) speech understandable by the users, in this case, people with language impairments.

Overview Paper

• Bogdan Ionescu, Henning Müller, Ana-Maria Drăgulinescu, Johannes Rückert, Asma Ben Abacha, Alba G. Seco de Herrera, Louise Bloch, Raphael Brüngel, Ahmad Idrissi-Yaghir, Henning Schäfer, Cynthia Sabrina Schmidt, Tabea M.G. Pakull, Hendrik Damm, Benjamin Bracke, Christoph M. Friedrich, Alexandra-Georgiana Andrei, Yuri Prokopchuk, Dzmitry Karpenka, Ahmedkhan Radzhabov, Vassili Kovalev, Cécile Macaire, Didier Schwab, Benjamin Lecouteux, Emmanuelle Esperança-Rodier, Wen-wai Yim, Yujuan Fu, Zhaoyi Sun, Meliha Yetisgen, Fei Xia, Steven A. Hicks, Michael A. Riegler, Vajira Thambawita, Andrea Storås, Pål Halvorsen, Maximilian Heinrich, Johannes Kiesel, Martin Potthast, Benno Stein, Overview of the ImageCLEF 2024: Multimedia Retrieval in Medical Applications, in Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the 15th International Conference of the CLEF Association (CLEF 2024), Springer Lecture Notes in Computer Science LNCS, Grenoble, France, 9-12 September, 2024.
• Latex:
  

  @inproceedings{ImageCLEF2024,
      author       = {Bogdan Ionescu and Henning M\"uller and Ana{-}Maria Dr\u{a}gulinescu and
                      Johannes R\"uckert and Asma {Ben Abacha} and Alba {Garc\’{\i}a Seco de
                      Herrera} and Louise Bloch and Raphael Br\"ungel and Ahmad Idrissi{-}Yaghir
                      and Henning Sch\"afer and Cynthia Sabrina Schmidt and Tabea M.G. Pakull and
                      Hendrik Damm and Benjamin Bracke and Christoph M. Friedrich and
                      Alexandra{-}Georgiana Andrei and Yuri Prokopchuk and Dzmitry Karpenka and
                      Ahmedkhan Radzhabov and Vassili Kovalev and C\'ecile Macaire and Didier
                      Schwab and Benjamin Lecouteux and Emmanuelle Esperan\c{c}a{-}Rodier and
                      Wen{-}wai Yim and Yujuan Fu and Zhaoyi Sun and Meliha Yetisgen and Fei Xia
                      and Steven A. Hicks and Michael A. Riegler and Vajira Thambawita and Andrea
                      Stor\r{a}s and P\r{a}l Halvorsen and Maximilian Heinrich and Johannes Kiesel
                      and Martin Potthast and Benno Stein},
      title        = {{Overview of ImageCLEF 2024}: Multimedia Retrieval in Medical Applications},
      booktitle    = {Experimental IR Meets Multilinguality, Multimodality, and Interaction},
      series       = {Proceedings of the 15th International Conference of the CLEF Association
                      (CLEF 2024)},
      year         = {2024},
      publisher    = {Springer Lecture Notes in Computer Science LNCS},
      pages        = {},
      month        = {September 9-12},
      address      = {Grenoble, France}
  }
  

The Organising Committee

Overall coordination

• Bogdan Ionescu <bogdan.ionescu(at)upb.ro>, National University of Science and Technology Politehnica Bucharest, Romania
• Henning Müller <henning.mueller(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland
• Ana-Maria Drăgulinescu <ana.dragulinescu(at)upb.ro>, National University of Science and Technology Politehnica Bucharest, Romania

Technical support

• Ivan Eggel <ivan.eggel(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland
• Liviu-Daniel Ștefan <liviu_daniel.stefan(at)upb.ro>, National University of Science and Technology Politehnica Bucharest, Romania