ImageCLEFmed Tuberculosis

Motivation

Welcome to the 4th edition of the Tuberculosis Task!

Tuberculosis (TB) is a bacterial infection caused by a germ called Mycobacterium tuberculosis. About 130 years after its discovery, the disease remains a persistent threat and a leading cause of death worldwide according to WHO. This bacteria usually attacks the lungs, but it can also damage other parts of the body. Generally, TB can be cured with antibiotics. However, the different types of TB require different treatments, and therefore the detection of the TB type and the evaluation of lesion characteristics are important real-world tasks.

Lessons learned:

• Tuberculosis task exists in ImageCLEF since 2017 and it was modified from year to year.
• In the first and second editions of this task, held at ImageCLEF 2017 and ImageCLEF 2018, participants had to detect Multi-drug resistant patients (MDR subtask) and to classify the TB type (TBT subtask) both based only on the CT image. After 2 editions we concluded that the MDR subtask was not possible to solve based only on the image. In the TBT subtask, there was a slight improvement in 2018 with respect to 2017 on the classification results, however, not enough considering the amount of extra data provided in the 2018 edition, both in terms of new images and meta-data.
• In the third edition Tuberculosis task was restructured to allow usage of uniform dataset, and included two subtasks - continued Severity Score (SVR) prediction subtask and a new subtask based on providing an automatic report (CT Report) on the TB case.
• In this year edition, we decided to concentrate on the automated CT report generation task, since it has important outcome that can have a major impact in the real-world clinical routines. In order to make the task both more attractive for participants and practically valuable, this year report generation is lung-based rather than CT-based, which means the labels for left and right lungs will be provided independently. The set of target labels in the CT Report was updated with accordance to the opinion of medical experts. This year we provide 3 labels for each lung: presence of TB lesions in general, presence of pleurisy and caverns in particular. Also the dataset size was increased compared to the previous year.

News

• 10.12.2019: Task web-page goes live
• 20.01.2020: Task web-page updated. Finishing competition page on AIcrowd
• 27.01.2020: Competition page available on AIcrowd

Task description

In this task participants have to generate automatic lung-wise reports based on the CT image data.
Each report should include the probability scores (ranging from 0 to 1) for each of the three labels and for each of the lungs (resulting in 6 entries per CT). The resulting list of entries includes: LeftLungAffected, RightLungAffected, CavernsLeft, CavernsRight, PleurisyLeft, PleurisyRight.

Data

In this edition, a dataset containing chest CT scans of 403 (283 for train and 120 for test) TB patients is used. Since the labels are provided on lung-wise scale rather than CT-wise scale, the total number of cases is virtually increased twice.

For all patients we provide 3D CT images with an image size per slice of 512*512 pixels and number of slices being around 100. All the CT images are stored in NIFTI file format with .nii.gz file extension (g-zipped .nii files). This file format stores raw voxel intensities in Hounsfield units (HU) as well the corresponding image metadata such as image dimensions, voxel size in physical units, slice thickness, etc. A freely-available tool called "VV" can be used for viewing the image files. Currently, there are various tools available for reading and writing NIFTI files. Among them there are load_nii and save_nii functions for Matlab and Niftilib library for C, Java, Matlab and Python, NiBabel package for Python.

Moreover, for all patients we provide two versions of automatically extracted masks of the lungs. These data can be downloaded together with the patients CT images.
The first version of segmentation was retrieved using the same technique as the previous years. The details of this segmentation can be found here.
The second version of segmentation was retrieved using non-rigid image registration scheme. The details of this segmentation and open-source implementation can be found here.
The first version of segmentation provides more accurate masks, but it tends to miss large abnormal regions of lungs in the most severe TB cases. The second segmentation on the contrary provides more rough bounds, but behaves more stable in terms of including lesion areas.
In case the participants use the provided masks in their experiments, please refer to the section "Citations" at the end of this page to find the appropriate citation for the corresponding lung segmentation technique.

Evaluation methodology

This task is considered as a multi-binary classification problem.
The ranking of this task will be done first by average AUC and then by min AUC over the 3 target labels.
The AUC values will be evaluated in a lung-wise manner.

Preliminary Schedule

• 22.01.2020: registration opens (until 27.04.2020)
• 22.01.2020: development data release starts
• 27.03.2020: test data release starts (tentative)
• 05.06.2020: deadline for submitting the participants runs
• 12.06.2020: release of the processed results by the task organizers
• 10.07.2020: deadline for submission of working notes papers by the participants
• 07.08.2020: notification of acceptance of the working notes papers
• 21.08.2020: camera ready working notes papers
• 22-25.09.2020: CLEF 2020, Thessaloniki, Greece - foreseen as a virtual conference

Participant registration

Please refer to the general ImageCLEF registration instructions

Submission instructions

Submit a plain text file named with the prefix CTR (e.g. CTRfree-text.txt) with the following format:

• <Filename>,<Probability of "left lung affected">,<Probability of "right lung affected">,<Probability of "presence of caverns in the left lung">,<Probability of "presence of caverns in the right lung">,<Probability of "pleurisy in the left lung">,<Probability of "pleurisy in the right lung">

e.g.:

• CTR_TST_001.nii.gz,0.89,0.1,0.84,0.05,0.9,0.2
• CTR_TST_002.nii.gz,0.1,0.6,0.222,0.333,0.444,0.55
• CTR_TST_003.nii.gz,0.1,0.7,0.0,0.2,0.1,0.46
• CTR_TST_004.nii.gz,0.88,0.78,0.59,0.65,0.8,0.4

You need to respect the following constraints:

• Filenames must be the same as the original test file names
• All filenames must be present in the runfiles
• Only use numbers between 0 and 1 for the probabilities. Use the dot (.) as a decimal point (no commas accepted)

Results

Citations

• When referring to the ImageCLEFtuberculosis 2020 task general goals, general results, etc. please cite the following publication (also referred to as ImageCLEF tuberculosis task overview):
  
  • Serge Kozlovski, Vitali Liauchuk, Yashin Dicente Cid, Aleh Tarasau, Vassili Kovalev, Henning Müller, Overview of ImageCLEFtuberculosis 2020 - Automatic CT-based Report Generation, CLEF working notes, CEUR, 2020.
  • BibTex:
    @Inproceedings{ImageCLEFTBoverview2020,
    author = {Kozlovski, Serge and Liauchuk, Vitali and Dicente Cid, Yashin and Tarasau, Aleh and Kovalev, Vassili and M\"uller, Henning},
    title = {Overview of {ImageCLEFtuberculosis} 2020 - Automatic {CT}-based Report Generation},
    booktitle = {CLEF2020 Working Notes},
    series = {{CEUR} Workshop Proceedings},
    year = {2020},
    volume = {},
    publisher = {CEUR-WS.org $<$http://ceur-ws.org$>$},
    pages = {},
    month = {September 22-25},
    address = {Thessaloniki, Greece}
    }
• When referring to the ImageCLEF 2020 lab general goals, general results, etc. please cite the following publication which will be published by September 2020 (also referred to as ImageCLEF general overview):
  
  • Bogdan Ionescu, Henning Müller, Renaud Péteri, Asma Ben Abacha, Vivek Datla, Sadid A. Hasan, Dina Demner-Fushman, Serge Kozlovski, Vitali Liauchuk, Yashin Dicente Cid, Vassili Kovalev, Obioma Pelka, Christoph M. Friedrich, Alba García Seco de Herrera, Van-Tu Ninh, Tu-Khiem Le, Liting Zhou, Luca Piras, Michael Riegler, Pål Halvorsen, Minh-Triet Tran, Mathias Lux, Cathal Gurrin, Duc-Tien Dang-Nguyen, Jon Chamberlain, Adrian Clark, Antonio Campello, Dimitri Fichou, Raul Berari, Paul Brie, Mihai Dogariu, Liviu Daniel Ștefan, Mihai Gabriel Constantin, Overview of the ImageCLEF 2020: Multimedia Retrieval in Medical, Lifelogging, Nature, and Internet Applications In: Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the 11th International Conference of the CLEF Association (CLEF 2020), Thessaloniki, Greece, LNCS Lecture Notes in Computer Science, Springer (September 22-25 2020)
  • BibTex:
    @inproceedings{ImageCLEF20,
    author = {Bogdan Ionescu and Henning M\"uller and Renaud P\'{e}teri
    and Asma Ben Abacha and Vivek Datla and Sadid A. Hasan and Dina
    Demner-Fushman and Serge Kozlovski and Vitali Liauchuk and Yashin
    Dicente Cid and Vassili Kovalev and Obioma Pelka and Christoph M.
    Friedrich and Alba Garc\'{\i}a Seco de Herrera and Van-Tu Ninh and
    Tu-Khiem Le and Liting Zhou and Luca Piras and Michael Riegler and
    P\aa l Halvorsen and Minh-Triet Tran and Mathias Lux and Cathal Gurrin
    and Duc-Tien Dang-Nguyen and Jon Chamberlain and Adrian Clark and
    Antonio Campello and Dimitri Fichou and Raul Berari and Paul Brie and
    Mihai Dogariu and Liviu Daniel \c{S}tefan and Mihai Gabriel
    Constantin},
    title = {{Overview of the ImageCLEF 2020}: Multimedia Retrieval in
    Medical, Lifelogging, Nature, and Internet Applications},
    booktitle = {Experimental IR Meets Multilinguality, Multimodality, and
    Interaction},
    series = {Proceedings of the 11th International Conference of the CLEF
    Association (CLEF 2020)},
    year = {2020},
    volume = {12260},
    publisher = {{LNCS} Lecture Notes in Computer Science, Springer},
    pages = {},
    month = {September 22-25},
    address = {Thessaloniki, Greece}
    }
• When using any of the provided mask of the lungs, please cite the corresponding publication:
  for segmentation #1 (fully automatic multistage):
  • Yashin Dicente Cid, Oscar A. Jiménez-del-Toro, Adrien Depeursinge, and Henning Müller. Efficient and fully automatic segmentation of the lungs in CT volumes. In: Goksel, O., et al. (eds.) Proceedings of the VISCERAL Challenge at ISBI. No. 1390 in CEUR Workshop Proceedings (Apr 2015)
  • BibTex:

    @inproceedings{DJD2015,

    Title = {Efficient and fully automatic segmentation of the lungs in CT volumes},
    Booktitle = {Proceedings of the {VISCERAL} Anatomy Grand Challenge at the 2015 {IEEE ISBI}},
    Author = {Dicente Cid, Yashin and Jim{\'{e}}nez del Toro, Oscar Alfonso and Depeursinge, Adrien and M{\"{u}}ller, Henning},
    Editor = {Goksel, Orcun and Jim{\'{e}}nez del Toro, Oscar Alfonso and Foncubierta-Rodr{\'{\i}}guez, Antonio and M{\"{u}}ller, Henning},
    Keywords = {CAD, lung segmentation, visceral-project},
    Month = may,
    Series = {CEUR Workshop Proceedings},
    Year = {2015},
    Pages = {31-35},
    Publisher = {CEUR-WS.org $<$http://ceur-ws.org$>$},
    Location = {New York, USA}
    }
for segmentation #2 (via non-rigid image registration):
• Liauchuk, V., Kovalev, V. ImageCLEF 2017: Supervoxels and co-occurrence for tuberculosis CT image classification. In: CLEF2017 Working Notes. CEUR Workshop Proceedings, Dublin, Ireland, CEUR-WS.org http://ceur-ws.org (September 11-14 2017) (URL: http://ceur-ws.org/Vol-1866/paper_146.pdf)
• BibTex:

  @inproceedings{CLEF2017WN,

  Title = {ImageCLEF 2017: Supervoxels and co-occurrence for tuberculosis CT image classification},
  Author = {Liauchuk, Vitali and Kovalev, Vassili},
  Keywords = {Supervoxels, Dictionary, Co-occurrence, Image Classification},
  Booktitle = {CLEF2017 Working Notes},
  Series = {{CEUR} Workshop Proceedings},
  Year = {2017},
  Publisher = {CEUR-WS.org $<$http://ceur-ws.org$>$},
  Month = {September 11-14},
  Address = {Dublin, Ireland}
  }

Organizers

• Serge Kozlovski <kozlovski.serge(at)gmail.com>, Institute for Informatics, Minsk, Belarus
• Vitali Liauchuk <vitali.liauchuk(at)gmail.com>, Institute for Informatics, Minsk, Belarus
• Yashin Dicente Cid <yashin.dicente(at)warwick.ac.uk>, University of Warwick, Coventry, England, UK
• Vassili Kovalev <vassili.kovalev(at)gmail.com>, Institute for Informatics, Minsk, Belarus
• Henning Müller <henning.mueller(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland