You are here

Home › ImageCLEF 2020

ImageCLEFcoral

Welcome to the 2nd edition of the Coral Task!

Description

Motivation

The increasing use of structure-from-motion photogrammetry for modelling large-scale environments from action cameras attached to drones has driven the next-generation of visualisation techniques that can be used in augmented and virtual reality headsets. It has also created a need to have such models labelled, with objects such as people, buildings, vehicles, terrain, etc. all essential for machine learning techniques to automatically identify as areas of interest and to label them appropriately. However, the complexity of the images makes impossible for human annotators to assess the contents of images on a large scale.
Advances in automatically annotating images for complexity and benthic composition have been promising, and we are interested in automatically identify areas of interest and to label them appropriately for monitoring coral reefs. Coral reefs are in danger of being lost within the next 30 years, and with them the ecosystems they support. This catastrophe will not only see the extinction of many marine species, but also create a humanitarian crisis on a global scale for the billions of humans who rely on reef services. By monitoring the changes and composition of coral reefs we can help prioritise conservation efforts.

If you participate in this task, you may want also to check the DrawnUI Task which addresses a similar classification problem, but in a different use case scenario. For more information see: https://www.imageclef.org/2020/drawnui

News

  • 22.04.2020: schedule uploaded
  • 18.04.2020: training test revised data uploaded
  • 04.04.2020: training data corrected
  • 01.04.2020: test data released

Preliminary Schedule

  • 21.01.2020: registration opens (until 27.04.2020)
  • 21.01.2020: development data released
  • 16.03.202004.01.2020: test data release starts
  • 11.05.2020,05.06.202008.06.2020: deadline for submitting the participants runs
  • 18.05.202012.06.2020: release of the processed results by the task organizers
  • 25.05.202010.07.2020: deadline for submission of working notes papers by the participants
  • 15.06.202007.08.2020: notification of acceptance of the working notes papers
  • 29.06.202021.08.2020: camera ready working notes papers
  • 22-25.09.2020: CLEF 2020, Thessaloniki, Greece

Coral reef image annotation and localisation task

This task requires the participants to label the images with types of benthic substrate together with their bounding box in the image. Each image is provided with possible class types. For each image, participants will produce a set of bounding boxes, predicting the benthic substrate for each bounding box in the image.

Coral reef image pixel-wise parsing task

This task requires the participants to segment and parse each coral reef image into different image regions associated with benthic substrate types. For each image, segmentation algorithms will produce a semantic segmentation mask, predicting the semantic category for each pixel in the image.

Data

The data for this task originates from a growing, large-scale collection of images taken from coral reefs around the world as part of a coral reef monitoring project with the Marine Technology Research Unit at the University of Essex.
Substrates of the same type can have very different morphologies, color variation and patterns. Some of the images contain a white line (scientific measurement tape) that may occlude part of the entity. The quality of the images is variable, some are blurry, and some have poor color balance. This is representative of the Marine Technology Research Unit dataset and all images are useful for data analysis. The images contain annotations of the following 13 types of substrates: Hard Coral – Branching, Hard Coral – Submassive, Hard Coral – Boulder, Hard Coral – Encrusting, Hard Coral – Table, Hard Coral – Foliose, Hard Coral – Mushroom, Soft Coral, Soft Coral – Gorgonian, Sponge, Sponge – Barrel, Fire Coral – Millepora and Algae - Macro or Leaves.
The test data contains images from four different locations:

  • same location as training set
  • similar location to training set
  • geographically similar to training set
  • geographically distinct from training set

Evaluation methodology

The evaluation will be carry out using the PASCAL style metric of intersection over union (IoU), the area of intersection between the foreground in the
output segmentation and the foreground in the ground-truth segmentation, divided by the area of their union.
The final results will be presented both in terms of average performance over all images of all concepts, and also per concept performance over all images.

MAP_0.5 Is the localised Mean average precision (MAP) for each submitted method for using the performance measure of IoU >=0.5 of the ground truth

MAP_0 Is the image annotation average for each method with success if the concept is simply detected in the image without any localisation


<\p>
You can find the code for the annotation and localisation task here.
You can find the code for the pixel-wise parsing task here.

Accuracy per substrate The segmentation accuracy for a substrate will be assessed using the number of correctly labelled pixels of that substrate, divided by the number of pixels labelled with that class (in either the ground truth labelling or the inferred labelling).
You can find the code to calculate the accuracy per substrate here.
You can find the code for the pixel-wise parsing task

Participant registration

Please refer to the general ImageCLEF registration instructions

Submission instructions

The submissions will be received through the crowdAI

system.
Participants will be permitted to submit up to 10 runs. External training data is allowed and encouraged.
Each system run will consist of a single ASCII plain text file. The results of each test set should be given in separate lines in the text file. The format of the text file is as follows:
[image_ID/document_ID] [results]

Coral reef image annotation and localisation

The results of each test set image should be given in separate lines, each line providing only up to 500 localised substrates. The format has characters to separate the elements, semicolon ‘;’ for the substrates, colon ':' for the confidence, comma ',' to separate multiple bounding boxes, and 'x' and '+' for the size-offset bounding box format, i.e.:

[image_ID];[substrate1] [[confidence1,1]:][width1,1]x[height1,1]+[xmin1,1]+[ymin1,1],[[confidence1,2]:][width1,2]x[height1,2]+[xmin1,2]+[ymin1,2],...;[substrate2] ...

[confidence] are floating point values 0-1 for which a higher value means a higher score.

For example, in the development set format (notice that there are 2 bounding boxes for substrate c_soft_coral):

  • 2018_0714_112604_057 0 c_hard_coral_branching 1 891 540 1757 1143
  • 2018_0714_112604_057 3 c_soft_coral 1 2724 1368 2825 1507
  • 2018_0714_112604_057 4 c_soft_coral 1 2622 1576 2777 1731

In the submission format, it would be a line as:

  • 2018_0714_112604_057;c_hard_coral_branching 0.6:867x 604+891+540;c_soft_coral 0.7:102x140+2724+2825,0.3:156x156+2622+1576

Coral reef image pixel-wise parsing task

Similar to subtask 1, the results of each test set image should be given in separate lines, each line providing only up to 500 localised substrates, with up to 500 coordinate localisations of the same substrate expected. The format has characters to separate the elements, semicolon ';' for the substrates, colon ':' for the confidence, comma ',' to separate multiple bounding polygons, and 'x' and '+' for the size-offset bounding polygon format, i.e.:

[image_ID];[substrate1] [[confidence1,1]:][x1,1]+[y1,1]+[x2,1]+[y2,1]+….+[xn,1]+[yn,1],[[confidence1,2][x1,2]+[y1,2]+[x2,2]+[y2,2]+….+[xn,2]+[yn,2];[substrate2] ...

[confidence] are floating point values 0-1 for which a higher value means a higher score and the [xi,yi] represents consecutive points.

For example, in the development set format (notice that there are 2 polygons for substrate c_soft_coral):

  • 2018_0714_112604_057 0 c_hard_coral_branching 1 1757 833 1645 705 1559 598 1442 540 1249 593 1121 679 1020 705 998 844 891 967 966 1122 1137 1143 1324 1122 1468 1074 1655 978
  • 2018_0714_112604_057 3 c_soft_coral 1 2804 1368 2745 1368 2724 1427 2729 1507 2809 1507 2825 1453
  • 2018_0714_112604_057 4 c_soft_coral 1 2697 1576 2638 1592 2638 1608 2622 1667 2654 1694 2713 1731 2777 1731 2777 1635

In the submission format, it would be a line as:

  • 2018_0714_112604_057;c_hard_coral_branching 0.6:1757+833+1645+705+1559+598+1442+540+1249+593+1121+679+1020+705+998+844+891+967+966+1122+1137+1143+1324+1122+1468+1074+1655+978;c_soft_coral 0.7:2804+1368+2745+1368+2724+1427+2729+1507+2809+1507+2825+1453,0.3:2697+1576+2638+1592+2638+1608+2622+1667+2654+1694+2713+1731+2777+1731+2777+1635

Results

Subtask #1: Coral reef image annotation and localisation task

The following table presents the MAP_0.5 and MAP_0.0 on the test set. The first two columns presents the results on all the test set followed by the results on each of the four different locations in the test set.

Run Group MAP_0.5 MAP_0.0 Same location MAP_0.5 Same location MAP_0.0 Similar location MAP_0.5 Similar location MAP_0.0 Geographically similar MAP_0.5 Geographically similar MAP_0.0 Geographically distinct MAP_0.5 Geographically distinct MAP_0.0
68143 FAV ZČU PiVa 0.582 0.853 0.648 0.876 0.343 0.806 0.627 0.97 0.329 0.629
67863 FAV ZČU PiVa 0.565 0.851 0.614 0.88 0.44 0.758 0.562 0.975 0.292 0.569
68094 FAV ZČU PiVa 0.53 0.825 0.566 0.841 0.431 0.764 0.516 0.947 0.346 0.645
68145 FAV ZČU PiVa 0.517 0.814 0.547 0.831 0.438 0.738 0.533 0.978 0.344 0.631
67539 FAV ZČU CV 0.49 0.822 0.533 0.874 0.25 0.647 0.644 0.946 0.443 0.615
68181 FHD 0.457 0.775 0.538 0.795 0.317 0.704 0.415 0.868 0.392 0.701
68188 FHD 0.44 0.725 0.561 0.761 0.318 0.643 0.352 0.861 0.367 0.613
67862 FAV ZČU PiVa 0.439 0.774 0.489 0.797 0.322 0.746 0.508 0.939 0.28 0.536
68187 FHD 0.424 0.729 0.542 0.762 0.294 0.654 0.35 0.861 0.359 0.619
68182 FHD 0.422 0.762 0.512 0.769 0.279 0.743 0.454 0.85 0.333 0.682
68146 FAV ZČU PiVa 0.415 0.747 0.475 0.775 0.258 0.684 0.527 0.947 0.242 0.551
68186 FHD 0.41 0.73 0.51 0.751 0.288 0.677 0.35 0.856 0.346 0.61
68183 FHD 0.405 0.759 0.508 0.767 0.253 0.718 0.399 0.905 0.363 0.663
68201 HHU 0.392 0.806 0.457 0.815 0.283 0.864 0.426 0.812 0.125 0.362
67914 FHD 0.391 0.72 0.441 0.711 0.296 0.677 0.382 0.922 0.303 0.621
68184 FHD 0.388 0.707 0.482 0.728 0.283 0.671 0.343 0.789 0.322 0.62
67919 FHD 0.383 0.703 0.454 0.706 0.268 0.629 0.416 0.905 0.248 0.588
68138 FAV ZČU PiVa 0.377 0.721 0.434 0.749 0.254 0.656 0.516 0.942 0.21 0.544
68185 FHD 0.369 0.722 0.477 0.754 0.231 0.686 0.341 0.819 0.274 0.584
67858 FAV ZČU PiVa 0.357 0.712 0.417 0.736 0.247 0.679 0.446 0.916 0.23 0.514
68093 FAV ZČU PiVa 0.349 0.709 0.41 0.732 0.23 0.666 0.453 0.912 0.231 0.547
67857 FAV ZČU PiVa 0.347 0.728 0.401 0.743 0.234 0.706 0.47 0.939 0.225 0.545
68202 HHU 0.323 0.753 0.339 0.764 0.222 0.708 0.435 0.915 0.257 0.58
68198 HHU 0.313 0.702 0.359 0.692 0.201 0.684 0.418 0.916 0.272 0.599
68205 HHU 0.303 0.727 0.337 0.779 0.187 0.527 0.269 0.761 0.2 0.523
68196 HHU 0.28 0.684 0.351 0.673 0.171 0.679 0.459 0.878 0.18 0.646
68212 HHU 0.263 0.663 0.312 0.644 0.165 0.67 0.39 0.895 0.208 0.555
68197 HHU 0.245 0.628 0.406 0.75 0.003 0.468 0.01 0.463 0 0.299
67558 FAV ZČU CV 0.243 0.664 0.326 0.672 0.123 0.687 0.245 0.832 0.144 0.473
68213 HHU 0.233 0.644 0.296 0.619 0.144 0.661 0.405 0.854 0.149 0.612
68178 HHU 0.01 0.206 0.013 0.196 0.004 0.221 0.004 0.222 0.012 0.231
68179 HHU 0.01 0.274 0.01 0.273 0.024 0.325 0 0.278 0 0

The following table presents the accuracy per substrate across all the test set:

Accuracy per substrate: overall
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68213 HHU 0.306 0.038 0.171 0.066 0.042 0.097 0.15 0.359 0.082 0.12 0.089 0 0.016 0.118
68212 HHU 0.327 0.067 0.218 0.09 0.034 0.105 0.231 0.445 0.059 0.121 0.134 0 0.012 0.142
68205 HHU 0.091 0.024 0.093 0.023 0.017 0.056 0.081 0.185 0 0.036 0.039 0 0.019 0.051
68202 HHU 0.247 0.032 0.16 0.053 0.026 0.115 0.154 0.314 0.037 0.067 0.082 0 0.007 0.1
68201 HHU 0.142 0.002 0.052 0.005 0 0.003 0.155 0.144 0 0.019 0.023 0 0.016 0.043
68198 HHU 0.316 0.041 0.221 0.077 0.029 0.119 0.183 0.462 0.037 0.107 0.115 0 0.002 0.131
68182 FHD 0.35 0 0.199 0.005 0 0.012 0.187 0.484 0 0.104 0.064 0 0 0.108
68183 FHD 0.337 0 0.209 0 0 0.022 0.197 0.456 0 0.07 0.098 0 0 0.107
68197 HHU 0.137 0.011 0.105 0.022 0.021 0.072 0.149 0.399 0 0.045 0.028 0 0.008 0.077
68196 HHU 0.3 0.015 0.159 0.056 0.022 0.056 0.128 0.367 0.088 0.123 0.053 0 0.005 0.105
68188 FHD 0.301 0.004 0.249 0.032 0.035 0.182 0.397 0.453 0.057 0.088 0.135 0 0.006 0.149
68187 FHD 0.306 0.007 0.256 0.038 0.039 0.192 0.402 0.48 0.061 0.099 0.134 0 0.009 0.155
68186 FHD 0.322 0 0.257 0.03 0.031 0.195 0.42 0.485 0.076 0.097 0.142 0 0.008 0.159
68185 FHD 0.319 0 0.185 0.039 0.02 0.131 0.156 0.423 0.046 0.084 0.119 0 0 0.117
68184 FHD 0.323 0.003 0.246 0.052 0.041 0.122 0.25 0.467 0.067 0.109 0.102 0 0.006 0.137
68181 FHD 0.316 0 0.179 0 0 0 0.288 0.472 0 0.12 0.02 0 0 0.107
68179 HHU 0.037 0 0 0 0 0 0 0.146 0 0 0 0 0 0.014
68178 HHU 0.014 0.002 0.017 0.005 0 0.001 0 0.056 0 0.009 0 0 0 0.008
68146 FAV ZČU PiVa 0.181 0.011 0.133 0.046 0 0.13 0.182 0.452 0 0.103 0.049 0 0 0.099
68145 FAV ZČU PiVa 0.123 0 0.105 0.021 0 0.038 0.122 0.387 0 0.093 0.014 0 0 0.069
68143 FAV ZČU PiVa 0.089 0.002 0.054 0.008 0 0.009 0.109 0.29 0 0.065 0.01 0 0 0.049
68138 FAV ZČU PiVa 0.211 0.016 0.159 0.052 0 0.149 0.204 0.462 0 0.113 0.062 0 0.001 0.11
68094 FAV ZČU PiVa 0.127 0.001 0.108 0.02 0 0.038 0.121 0.393 0 0.087 0.004 0 0 0.069
68093 FAV ZČU PiVa 0.3 0.017 0.206 0.08 0.009 0.147 0.22 0.465 0.082 0.117 0.067 0 0.001 0.132
67919 FHD 0.243 0 0.222 0.049 0 0 0 0.45 0 0.118 0 0 0 0.083
67914 FHD 0.259 0 0.227 0.042 0 0.086 0.194 0.474 0 0.13 0.057 0 0 0.113
67863 FAV ZČU PiVa 0.104 0.002 0.103 0.01 0 0.001 0.134 0.338 0 0.07 0.004 0 0 0.059
67862 FAV ZČU PiVa 0.219 0.006 0.176 0.038 0.008 0.101 0.211 0.464 0.033 0.106 0.03 0 0 0.107
67858 FAV ZČU PiVa 0.297 0.032 0.22 0.057 0.012 0.1 0.315 0.508 0.047 0.11 0.089 0 0 0.138
67857 FAV ZČU PiVa 0.306 0.034 0.221 0.06 0.015 0.105 0.32 0.512 0.044 0.111 0.09 0 0 0.14
67558 FAV ZČU CV 0.228 0 0.216 0.048 0 0.031 0.139 0.413 0.026 0.097 0.064 0 0 0.097
67539 FAV ZČU CV 0.259 0 0.155 0.047 0.015 0.094 0.096 0.475 0.028 0.057 0.085 0 0 0.101

The following tables present the accuracy per substrate across the subsets in the test set based on different locations

Accuracy per substrate: Same location
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68186 FHD 0.273 0 0.418 0.061 0.178 0.398 0.406 0.494 0.148 0.167 0.487 0 0.249 0.252
68187 FHD 0.28 0.002 0.424 0.084 0.246 0.389 0.386 0.487 0.149 0.143 0.31 0 0.225 0.24
67858 FAV ZČU PiVa 0.31 0.113 0.442 0.139 0.145 0.32 0.302 0.508 0.208 0.222 0.272 0 0 0.229
67857 FAV ZČU PiVa 0.303 0.097 0.447 0.144 0.146 0.333 0.308 0.512 0.175 0.221 0.268 0 0 0.227
68188 FHD 0.28 0 0.423 0.072 0.222 0.386 0.372 0.452 0.121 0.126 0.312 0 0.178 0.226
68184 FHD 0.294 0 0.404 0.079 0.21 0.291 0.259 0.47 0.141 0.196 0.36 0 0.119 0.217
68093 FAV ZČU PiVa 0.316 0.008 0.429 0.146 0.145 0.355 0.241 0.49 0.194 0.186 0.115 0 0.04 0.205
68185 FHD 0.387 0 0.371 0.088 0.292 0.373 0.189 0.457 0.131 0.146 0.12 0 0 0.197
67539 FAV ZČU CV 0.38 0 0.443 0.144 0.131 0.293 0.143 0.49 0.177 0.146 0.168 0 0 0.193
67862 FAV ZČU PiVa 0.284 0.007 0.44 0.087 0.145 0.341 0.234 0.493 0.201 0.206 0.052 0 0 0.192
68138 FAV ZČU PiVa 0.286 0.026 0.446 0.099 0 0.349 0.245 0.465 0 0.188 0.225 0 0.061 0.184
68198 HHU 0.312 0.021 0.434 0.089 0.111 0.161 0.157 0.453 0.129 0.205 0.27 0 0 0.18
68197 HHU 0.274 0.031 0.399 0.04 0.309 0.21 0.088 0.408 0 0.155 0.247 0 0 0.166
68146 FAV ZČU PiVa 0.267 0.02 0.448 0.098 0 0.361 0.207 0.457 0 0.185 0.115 0 0 0.166
68182 FHD 0.355 0 0.429 0.013 0 0.159 0.211 0.5 0 0.193 0.276 0 0 0.164
67914 FHD 0.263 0 0.4 0.114 0 0.172 0.234 0.486 0 0.225 0.225 0 0 0.163
68212 HHU 0.274 0.037 0.387 0.095 0.086 0.165 0.138 0.428 0.057 0.201 0.117 0 0 0.153
68183 FHD 0.349 0 0.396 0 0 0.09 0.213 0.467 0 0.123 0.309 0 0 0.15
68213 HHU 0.211 0.084 0.287 0.121 0.047 0.094 0.18 0.342 0.111 0.139 0.151 0 0.078 0.142
68196 HHU 0.29 0.011 0.317 0.112 0.046 0.052 0.169 0.365 0.133 0.156 0.19 0 0 0.142
68202 HHU 0.208 0.008 0.217 0.07 0.11 0.164 0.147 0.347 0.133 0.122 0.281 0 0.006 0.14
68181 FHD 0.352 0 0.411 0 0 0 0.248 0.484 0 0.212 0.001 0 0 0.131
67558 FAV ZČU CV 0.233 0 0.321 0.11 0 0.071 0.109 0.431 0.172 0.078 0.122 0 0 0.127
68145 FAV ZČU PiVa 0.251 0 0.41 0.054 0 0.066 0.18 0.439 0 0.141 0 0 0 0.119
68094 FAV ZČU PiVa 0.251 0 0.422 0.052 0 0.066 0.182 0.434 0 0.138 0 0 0 0.119
67919 FHD 0.291 0 0.37 0.09 0 0 0 0.455 0 0.21 0 0 0 0.109
67863 FAV ZČU PiVa 0.122 0 0.393 0.036 0 0 0.199 0.375 0 0.102 0 0 0 0.094
68205 HHU 0.122 0.023 0.115 0.047 0.133 0.037 0.113 0.241 0 0.065 0.201 0 0 0.084
68143 FAV ZČU PiVa 0.176 0 0.281 0.007 0 0.05 0.159 0.301 0 0.08 0 0 0 0.081
68201 HHU 0.161 0.025 0.151 0.009 0 0.003 0.207 0.174 0 0.055 0.072 0 0.005 0.066
68179 HHU 0.013 0 0 0 0 0 0 0.16 0 0 0 0 0 0.013
68178 HHU 0.015 0.004 0.013 0.006 0 0 0 0.064 0 0.028 0 0 0 0.01
Accuracy per substrate: Similar location
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68184 FHD 0.247 0.009 0.153 0.031 0.029 0 0 0 0 0.08 0 0 0 0.042
68182 FHD 0.31 0 0.191 0 0 0 0 0 0 0.029 0 0 0 0.041
68181 FHD 0.266 0 0.163 0 0 0 0 0 0 0.074 0 0 0 0.039
68183 FHD 0.312 0 0.176 0 0 0 0 0 0 0.012 0 0 0 0.038
68185 FHD 0.264 0 0.127 0.039 0.007 0.015 0 0 0 0.025 0 0 0 0.037
68093 FAV ZČU PiVa 0.163 0.028 0.163 0.101 0.004 0 0 0 0 0.025 0 0 0 0.037
68186 FHD 0.195 0 0.187 0.006 0.016 0.011 0 0 0 0.023 0 0 0 0.034
67914 FHD 0.219 0 0.147 0.003 0 0 0 0.002 0 0.07 0 0 0 0.034
68187 FHD 0.181 0.01 0.187 0.003 0.027 0 0 0 0 0.022 0 0 0 0.033
68188 FHD 0.183 0.01 0.18 0.002 0.025 0 0 0 0 0.023 0 0 0 0.032
68196 HHU 0.26 0.016 0.084 0.013 0.004 0 0 0 0 0.014 0 0 0 0.03
67919 FHD 0.143 0 0.173 0.007 0 0 0 0.001 0 0.061 0 0 0 0.03
68213 HHU 0.233 0.033 0.061 0.024 0.002 0 0 0 0 0.012 0 0 0 0.028
68212 HHU 0.18 0.056 0.083 0.024 0.003 0 0 0 0 0.016 0 0 0 0.028
68198 HHU 0.188 0.044 0.089 0.019 0 0 0 0 0 0.013 0 0 0 0.027
67858 FAV ZČU PiVa 0.139 0.035 0.125 0.002 0.008 0 0 0 0 0.011 0 0 0 0.025
68202 HHU 0.178 0.039 0.063 0.015 0 0 0 0 0 0.011 0 0 0 0.024
67857 FAV ZČU PiVa 0.135 0.039 0.12 0.002 0.01 0 0 0 0 0.011 0 0 0 0.024
68138 FAV ZČU PiVa 0.079 0.014 0.121 0.045 0 0.01 0 0 0 0.02 0 0 0 0.022
68146 FAV ZČU PiVa 0.056 0.011 0.11 0.041 0 0.012 0 0 0 0.024 0 0 0 0.02
67862 FAV ZČU PiVa 0.062 0.012 0.137 0.002 0 0 0 0 0 0.021 0 0 0 0.018
67558 FAV ZČU CV 0.146 0 0.066 0.011 0 0 0 0 0 0.012 0 0 0 0.018
67539 FAV ZČU CV 0.119 0 0.055 0.008 0.005 0 0 0 0 0.029 0 0 0 0.017
68201 HHU 0.167 0 0.027 0 0 0 0 0 0 0.009 0 0 0.008 0.016
68197 HHU 0.116 0.003 0.042 0.013 0.001 0 0 0 0 0.023 0 0 0 0.015
68205 HHU 0.063 0.023 0.051 0.005 0 0 0 0 0 0.009 0 0 0 0.012
68179 HHU 0.127 0 0 0 0 0 0 0.001 0 0 0 0 0 0.01
68145 FAV ZČU PiVa 0.014 0 0.066 0 0 0 0 0 0 0.035 0 0 0 0.009
67863 FAV ZČU PiVa 0.002 0.005 0.098 0 0 0 0 0 0 0.013 0 0 0 0.009
68094 FAV ZČU PiVa 0.015 0.002 0.059 0 0 0 0 0 0 0.033 0 0 0 0.008
68178 HHU 0.022 0.003 0.027 0.008 0 0 0 0 0 0.005 0 0 0 0.005
68143 FAV ZČU PiVa 0.015 0.006 0.016 0.002 0 0 0 0 0 0.026 0 0 0 0.005
Accuracy per substrate: Geographically similar
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68186 FHD 0.588 0 0.246 0 0 0.016 0 0 0 0.01 0 0 0 0.066
68188 FHD 0.593 0 0.236 0 0 0.005 0 0 0 0.009 0 0 0 0.065
68187 FHD 0.564 0 0.253 0 0 0.006 0 0 0 0.009 0 0 0 0.064
68184 FHD 0.552 0 0.232 0 0 0.001 0 0 0 0.003 0 0 0 0.061
67858 FAV ZČU PiVa 0.495 0 0.25 0 0 0.018 0 0 0 0.001 0 0 0 0.059
68183 FHD 0.524 0 0.218 0 0 0 0 0.002 0 0.007 0 0 0 0.058
67857 FAV ZČU PiVa 0.481 0 0.25 0 0 0.018 0 0 0 0.001 0 0 0 0.058
68182 FHD 0.557 0 0.181 0 0 0 0 0.002 0 0.003 0 0 0 0.057
68093 FAV ZČU PiVa 0.515 0 0.222 0 0 0.006 0 0 0 0.001 0 0 0 0.057
67539 FAV ZČU CV 0.59 0 0.11 0 0 0.033 0 0 0 0 0 0 0 0.056
67862 FAV ZČU PiVa 0.484 0 0.184 0 0 0.018 0 0 0 0.002 0 0 0 0.053
68185 FHD 0.525 0 0.155 0 0 0 0 0 0 0 0 0 0 0.052
68138 FAV ZČU PiVa 0.464 0 0.161 0 0 0 0 0 0 0.002 0 0 0 0.048
68198 HHU 0.38 0 0.237 0 0 0 0 0 0 0 0 0 0 0.047
67914 FHD 0.339 0 0.238 0 0 0.011 0 0 0 0.003 0 0 0 0.046
68212 HHU 0.35 0 0.22 0.004 0 0 0 0 0 0 0 0 0 0.044
68196 HHU 0.488 0 0.09 0 0 0 0 0 0 0 0 0 0 0.044
68213 HHU 0.46 0 0.091 0 0 0 0 0 0 0 0 0 0 0.042
68202 HHU 0.33 0 0.217 0 0 0 0 0 0 0 0 0 0 0.042
68146 FAV ZČU PiVa 0.413 0 0.129 0 0 0 0 0 0 0.002 0 0 0 0.042
67919 FHD 0.267 0 0.271 0 0 0 0 0 0 0.005 0 0 0 0.042
68181 FHD 0.323 0 0.187 0 0 0 0 0 0 0.001 0 0 0 0.039
67558 FAV ZČU CV 0.177 0 0.264 0 0 0 0 0.001 0 0.002 0 0 0 0.034
67863 FAV ZČU PiVa 0.298 0 0.082 0 0 0.007 0 0 0 0.007 0 0 0 0.03
68094 FAV ZČU PiVa 0.28 0 0.08 0 0 0.001 0 0 0 0 0 0 0 0.028
68145 FAV ZČU PiVa 0.261 0 0.078 0 0 0.001 0 0 0 0.006 0 0 0 0.027
68143 FAV ZČU PiVa 0.206 0 0.094 0 0 0 0 0 0 0.004 0 0 0 0.023
68205 HHU 0.1 0 0.1 0 0 0 0 0 0 0 0 0 0 0.015
68201 HHU 0.146 0 0.033 0 0 0 0 0 0 0 0 0 0.002 0.014
68197 HHU 0.046 0 0.072 0 0 0 0 0 0 0 0 0 0 0.009
68179 HHU 0.044 0 0 0 0 0 0 0 0 0 0 0 0 0.003
68178 HHU 0.002 0 0.027 0 0 0.003 0 0 0 0 0 0 0 0.002
Accuracy per substrate: Geographically distinct
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
,68213, ,HHU, ,0, ,0, ,0.046, ,0.023, ,0, ,0, ,0, ,0, ,0, ,0.168, ,0.063, ,0, ,0.017, ,0,
,68212, ,HHU, ,0, ,0, ,0.11, ,0.043, ,0, ,0, ,0, ,0, ,0, ,0.124, ,0.065, ,0, ,0.004, ,0,
,68205, ,HHU, ,0, ,0, ,0.084, ,0.012, ,0, ,0, ,0, ,0, ,0, ,0.038, ,0.018, ,0, ,0.042, ,0,
,68202, ,HHU, ,0, ,0, ,0.122, ,0.018, ,0, ,0, ,0, ,0, ,0, ,0.063, ,0.042, ,0, ,0.001, ,0,
,68201, ,HHU, ,0, ,0, ,0.002, ,0.014, ,0, ,0, ,0, ,0, ,0, ,0.007, ,0.006, ,0, ,0.019, ,0,
,68198, ,HHU, ,0, ,0, ,0.153, ,0.047, ,0, ,0, ,0, ,0, ,0, ,0.101, ,0.063, ,0, ,0, ,0,
,68182, ,FHD, ,0, ,0, ,0.092, ,0.001, ,0, ,0, ,0, ,0, ,0, ,0.06, ,0.053, ,0, ,0, ,0,
,68183, ,FHD, ,0, ,0, ,0.081, ,0, ,0, ,0, ,0, ,0, ,0, ,0.056, ,0.059, ,0, ,0, ,0,
,68197, ,HHU, ,0, ,0, ,0, ,0.001, ,0, ,0, ,0, ,0, ,0, ,0.009, ,0, ,0, ,0.02, ,0,
,68196, ,HHU, ,0, ,0, ,0.104, ,0.025, ,0, ,0, ,0, ,0, ,0, ,0.151, ,0.051, ,0, ,0.005, ,0,
,68188, ,FHD, ,0, ,0, ,0.171, ,0.008, ,0, ,0, ,0, ,0, ,0, ,0.045, ,0.076, ,0, ,0, ,0,
,68187, ,FHD, ,0, ,0, ,0.154, ,0.009, ,0, ,0, ,0, ,0, ,0, ,0.058, ,0.074, ,0, ,0, ,0,
,68186, ,FHD, ,0, ,0, ,0.148, ,0.008, ,0, ,0, ,0, ,0, ,0, ,0.053, ,0.065, ,0, ,0, ,0,
,68185, ,FHD, ,0, ,0, ,0.14, ,0.009, ,0, ,0, ,0, ,0, ,0, ,0.057, ,0.091, ,0, ,0, ,0,
,68184, ,FHD, ,0, ,0, ,0.136, ,0.01, ,0, ,0, ,0, ,0, ,0, ,0.068, ,0.054, ,0, ,0.003, ,0,
,68181, ,FHD, ,0, ,0, ,0.091, ,0, ,0, ,0, ,0, ,0, ,0, ,0.06, ,0.042, ,0, ,0, ,0,
,68179, ,HHU, ,0, ,0, ,0, ,0, ,0, ,0, ,0, ,0, ,0, ,0, ,0, ,0, ,0, ,0,
,68178, ,HHU, ,0, ,0, ,0.001, ,0, ,0, ,0, ,0, ,0, ,0, ,0.008, ,0, ,0, ,0, ,0,
,68146, ,FAV ZČU PiVa, ,0, ,0, ,0.035, ,0.013, ,0, ,0, ,0, ,0, ,0, ,0.057, ,0.012, ,0, ,0, ,0,
,68145, ,FAV ZČU PiVa, ,0, ,0, ,0.013, ,0, ,0, ,0, ,0, ,0, ,0, ,0.029, ,0.006, ,0, ,0, ,0,
,68143, ,FAV ZČU PiVa, ,0, ,0, ,0, ,0.002, ,0, ,0, ,0, ,0, ,0, ,0.026, ,0.006, ,0, ,0, ,0,
,68138, ,FAV ZČU PiVa, ,0, ,0, ,0.03, ,0.013, ,0, ,0, ,0, ,0, ,0, ,0.086, ,0.015, ,0, ,0, ,0,
,68094, ,FAV ZČU PiVa, ,0, ,0, ,0.012, ,0, ,0, ,0, ,0, ,0, ,0, ,0.027, ,0.001, ,0, ,0, ,0,
,68093, ,FAV ZČU PiVa, ,0, ,0, ,0.029, ,0.018, ,0, ,0, ,0, ,0, ,0, ,0.088, ,0.022, ,0, ,0, ,0,
,67919, ,FHD, ,0, ,0, ,0.144, ,0.014, ,0, ,0, ,0, ,0, ,0, ,0.067, ,0, ,0, ,0, ,0,
,67914, ,FHD, ,0, ,0, ,0.19, ,0.008, ,0, ,0, ,0, ,0, ,0, ,0.072, ,0.016, ,0, ,0, ,0,
,67863, ,FAV ZČU PiVa, ,0, ,0, ,0.01, ,0, ,0, ,0, ,0, ,0, ,0, ,0.023, ,0.001, ,0, ,0, ,0,
,67862, ,FAV ZČU PiVa, ,0, ,0, ,0.056, ,0.009, ,0, ,0, ,0, ,0, ,0, ,0.05, ,0.01, ,0, ,0, ,0,
,67858, ,FAV ZČU PiVa, ,0, ,0, ,0.051, ,0.013, ,0, ,0, ,0, ,0, ,0, ,0.082, ,0.04, ,0, ,0, ,0,
,67857, ,FAV ZČU PiVa, ,0, ,0, ,0.046, ,0.013, ,0, ,0, ,0, ,0, ,0, ,0.087, ,0.045, ,0, ,0, ,0,
,67558, ,FAV ZČU CV, ,0, ,0, ,0.172, ,0.01, ,0, ,0, ,0, ,0, ,0, ,0.073, ,0.043, ,0, ,0, ,0,
,67539, ,FAV ZČU CV, ,0, ,0, ,0.169, ,0.035, ,0, ,0, ,0, ,0, ,0, ,0.023, ,0.019, ,0, ,0, ,0,

Subtask #2: Coral reef image pixel-wise parsing task

The following table presents the MAP_0.5 and MAP_0.0 on the test set. The first two columns presents the results on all the test set followed by the results on each of the four different locations in the test set.

Run Group MAP_0.5 MAP_0.0 Same location MAP_0.5 Same location MAP_0.0 Similar location MAP_0.5 Similar location MAP_0.0 Geographically similar MAP_0.5 Geographically similar MAP_0.0 Geographically distinct MAP_0.5 Geographically distinct MAP_0.0
67864 FAV ZČU PiVa 0.678 0.845 0.744 0.875 0.516 0.758 0.588 0.975 0.403 0.542
68139 FAV ZČU PiVa 0.664 0.842 0.742 0.874 0.448 0.776 0.627 0.97 0.386 0.571
68095 FAV ZČU PiVa 0.629 0.817 0.67 0.835 0.553 0.756 0.537 0.947 0.439 0.617
68142 FAV ZČU PiVa 0.624 0.813 0.663 0.832 0.529 0.727 0.586 0.966 0.446 0.643
68144 FAV ZČU PiVa 0.617 0.807 0.656 0.826 0.546 0.731 0.556 0.978 0.418 0.607
68147 FAV ZČU PiVa 0.507 0.727 0.583 0.764 0.34 0.635 0.573 0.947 0.291 0.515
68190 FHD 0.474 0.715 0.597 0.758 0.349 0.606 0.401 0.89 0.379 0.558
68137 FAV ZČU PiVa 0.47 0.701 0.545 0.737 0.335 0.611 0.553 0.942 0.266 0.518
67968 FHD 0.469 0.708 0.595 0.757 0.337 0.596 0.359 0.859 0.411 0.593
67965 FHD 0.453 0.72 0.546 0.737 0.358 0.683 0.338 0.845 0.385 0.578
67964 FHD 0.449 0.717 0.542 0.726 0.333 0.649 0.366 0.85 0.425 0.639
67856 FAV ZČU PiVa 0.441 0.694 0.527 0.721 0.312 0.646 0.476 0.912 0.276 0.496
67967 FHD 0.435 0.695 0.519 0.718 0.331 0.633 0.383 0.898 0.341 0.572
68092 FAV ZČU PiVa 0.434 0.689 0.513 0.716 0.309 0.632 0.493 0.909 0.283 0.526
67963 FHD 0.433 0.694 0.493 0.702 0.348 0.616 0.404 0.92 0.334 0.552
68192 FHD 0.424 0.668 0.549 0.708 0.305 0.59 0.342 0.829 0.336 0.541
68191 FHD 0.416 0.692 0.541 0.749 0.26 0.563 0.335 0.867 0.323 0.53
68140 FAV ZČU PiVa 0.407 0.675 0.48 0.701 0.284 0.605 0.493 0.917 0.267 0.532
67969 FHD 0.376 0.629 0.504 0.668 0.257 0.548 0.293 0.81 0.338 0.507
68189 FHD 0.371 0.632 0.491 0.665 0.278 0.605 0.314 0.793 0.259 0.49
67620 FAV ZČU CV 0.304 0.602 0.409 0.637 0.187 0.537 0.243 0.79 0.199 0.414

The following table presents the accuracy per substrate across all the test set

Accuracy per substrate: overall
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68192 FHD 0.387 0.009 0.305 0.092 0.075 0.223 0.505 0.545 0.023 0.13 0.175 0 0.01 0.191
68191 FHD 0.333 0 0.222 0.009 0.021 0.132 0.255 0.49 0 0.085 0.116 0 0 0.128
68190 FHD 0.362 0 0.296 0.009 0.051 0.11 0.456 0.52 0.018 0.086 0.147 0 0 0.158
68189 FHD 0.338 0.003 0.294 0.072 0.059 0.124 0.245 0.522 0.061 0.133 0.177 0 0.01 0.157
68147 FAV ZČU PiVA 0.184 0.009 0.135 0.044 0 0.129 0.184 0.453 0 0.095 0.049 0 0 0.099
68144 FAV ZČU PiVA 0.125 0 0.109 0.02 0 0.041 0.122 0.394 0 0.092 0.014 0 0 0.071
68142 FAV ZČU PiVA 0.123 0 0.106 0.019 0 0.04 0.139 0.403 0 0.087 0.014 0 0 0.072
68140 FAV ZČU PiVA 0.283 0.012 0.203 0.075 0.01 0.148 0.226 0.443 0.055 0.113 0.079 0 0.001 0.127
68139 FAV ZČU PiVA 0.091 0.001 0.057 0.007 0 0.007 0.108 0.305 0 0.06 0.01 0 0 0.05
68137 FAV ZČU PiVA 0.213 0.013 0.162 0.05 0 0.148 0.199 0.456 0 0.102 0.064 0 0.001 0.108
68095 FAV ZČU PiVA 0.128 0 0.113 0.019 0 0.041 0.121 0.403 0 0.085 0.004 0 0 0.07
68092 FAV ZČU PiVA 0.293 0.013 0.21 0.077 0.01 0.128 0.225 0.462 0.055 0.109 0.071 0 0.001 0.127
67969 FHD 0.382 0.019 0.321 0.093 0.087 0.275 0.45 0.527 0.074 0.14 0.171 0 0.008 0.196
67968 FHD 0.342 0.006 0.307 0.043 0.048 0.213 0.435 0.544 0.047 0.113 0.158 0 0.009 0.174
67967 FHD 0.311 0 0.249 0.018 0 0.073 0.177 0.517 0 0.111 0.104 0 0 0.12
67965 FHD 0.296 0 0.286 0.014 0 0.102 0.226 0.522 0 0.105 0.11 0 0 0.128
67964 FHD 0.398 0 0.297 0.011 0 0.073 0.318 0.533 0 0.125 0.071 0 0 0.14
67963 FHD 0.303 0 0.276 0.058 0 0.149 0.19 0.538 0.05 0.16 0.018 0 0 0.134
67864 FAV ZČU PiVA 0.108 0 0.105 0.01 0 0.001 0.137 0.349 0 0.067 0.004 0 0 0.06
67856 FAV ZČU PiVA 0.287 0.026 0.228 0.055 0.014 0.104 0.318 0.498 0.051 0.099 0.091 0 0 0.136
67620 FAV ZČU CV 0.222 0 0.212 0.046 0 0.033 0.138 0.434 0.023 0.094 0.064 0 0 0.097

The following tables present the accuracy per substrate across the subsets in the test set based on different locations

Accuracy per substrate: Same location
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68192 FHD 0.359 0.071 0.539 0.192 0.365 0.406 0.473 0.556 0.15 0.242 0.462 0 0.208 0.309
68191 FHD 0.397 0 0.462 0.031 0.121 0.373 0.295 0.504 0 0.091 0.252 0 0 0.194
68190 FHD 0.316 0 0.52 0.016 0.262 0.289 0.496 0.509 0.007 0.085 0.378 0 0 0.221
68189 FHD 0.375 0 0.547 0.171 0.417 0.312 0.253 0.541 0.169 0.206 0.562 0 0.251 0.293
68147 FAV ZČU PiVa 0.292 0.019 0.486 0.106 0 0.358 0.239 0.461 0 0.171 0.133 0 0 0.174
68144 FAV ZČU PiVa 0.278 0 0.417 0.055 0 0.065 0.188 0.469 0 0.14 0 0 0 0.124
68142 FAV ZČU PiVa 0.278 0 0.448 0.053 0 0.065 0.214 0.464 0 0.129 0 0 0 0.127
68140 FAV ZČU PiVa 0.338 0.007 0.48 0.156 0.132 0.351 0.251 0.455 0.161 0.161 0.261 0 0.027 0.214
68139 FAV ZČU PiVa 0.196 0 0.292 0.007 0 0.043 0.163 0.337 0 0.08 0 0 0 0.086
68137 FAV ZČU PiVa 0.311 0.021 0.481 0.106 0 0.342 0.261 0.456 0 0.179 0.26 0 0.042 0.189
68095 FAV ZČU PiVa 0.278 0 0.431 0.053 0 0.065 0.191 0.464 0 0.137 0 0 0 0.125
68092 FAV ZČU PiVa 0.338 0.007 0.474 0.15 0.132 0.323 0.273 0.502 0.184 0.17 0.134 0 0.027 0.209
67969 FHD 0.329 0.058 0.517 0.146 0.34 0.405 0.47 0.542 0.339 0.215 0.374 0 0.135 0.298
67968 FHD 0.342 0.002 0.542 0.095 0.224 0.42 0.48 0.552 0.169 0.162 0.371 0 0.18 0.272
67967 FHD 0.38 0 0.515 0.037 0 0.221 0.237 0.532 0 0.191 0.267 0 0 0.183
67965 FHD 0.368 0 0.533 0.037 0 0.343 0.279 0.534 0 0.207 0.293 0 0 0.2
67964 FHD 0.409 0 0.526 0.018 0 0.314 0.365 0.542 0 0.2 0.172 0 0 0.196
67963 FHD 0.402 0 0.518 0.1 0 0.341 0.206 0.547 0.422 0.226 0.061 0 0 0.217
67864 FAV ZČU PiVa 0.142 0 0.41 0.036 0 0 0.208 0.409 0 0.105 0 0 0 0.101
67856 FAV ZČU PiVa 0.327 0.099 0.5 0.142 0.132 0.313 0.341 0.5 0.273 0.203 0.291 0 0 0.24
67620 FAV ZČU CV 0.264 0 0.427 0.113 0 0.068 0.107 0.46 0.191 0.078 0.115 0 0 0.14
Accuracy per substrate: Similar location
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68192 FHD 0.379 0.002 0.317 0.068 0.052 0 0 0 0 0.018 0 0 0 0.064
68191 FHD 0.354 0 0.198 0 0.013 0 0 0 0 0.013 0 0 0 0.045
68190 FHD 0.364 0 0.225 0 0.048 0.014 0 0 0 0.004 0 0 0 0.05
68189 FHD 0.285 0.008 0.297 0.145 0.044 0 0 0 0 0.119 0 0 0 0.069
68147 FAV ZČU PiVa 0.073 0.009 0.135 0.035 0 0.013 0 0 0 0.019 0 0 0 0.022
68144 FAV ZČU PiVa 0.023 0 0.072 0 0 0 0 0 0 0.048 0 0 0 0.011
68142 FAV ZČU PiVa 0.023 0 0.058 0 0 0 0 0 0 0.039 0 0 0 0.009
68140 FAV ZČU PiVa 0.202 0.018 0.212 0.088 0.004 0 0 0 0 0.02 0 0 0 0.042
68139 FAV ZČU PiVa 0.02 0.005 0.023 0 0 0 0 0 0 0.028 0 0 0 0.006
68137 FAV ZČU PiVa 0.094 0.011 0.159 0.045 0 0.011 0 0 0 0.015 0 0 0 0.026
68095 FAV ZČU PiVa 0.023 0.002 0.072 0 0 0 0 0 0 0.043 0 0 0 0.011
68092 FAV ZČU PiVa 0.2 0.021 0.228 0.096 0.004 0 0 0 0 0.021 0 0 0 0.044
67969 FHD 0.295 0.023 0.319 0.054 0.087 0 0 0 0 0.029 0 0 0 0.062
67968 FHD 0.241 0.008 0.304 0.001 0.038 0 0 0 0 0.033 0 0 0 0.048
67967 FHD 0.346 0 0.261 0.004 0 0 0 0 0 0.013 0 0 0 0.048
67965 FHD 0.343 0 0.3 0 0 0 0 0.003 0 0.023 0 0 0 0.051
67964 FHD 0.44 0 0.307 0.002 0 0 0 0 0 0.105 0 0 0 0.066
67963 FHD 0.305 0 0.222 0.029 0 0 0 0 0 0.075 0 0 0 0.049
67864 FAV ZČU PiVa 0.007 0.002 0.094 0 0 0 0 0 0 0.016 0 0 0 0.009
67856 FAV ZČU PiVa 0.175 0.024 0.171 0.002 0.01 0 0 0 0 0.01 0 0 0 0.03
67620 FAV ZČU CV 0.16 0 0.097 0.006 0 0 0 0 0 0.015 0 0 0 0.021
Accuracy per substrate: Geographically similar
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68192 FHD 0.674 0 0.345 0 0 0 0 0 0 0.008 0 0 0 0.079
68191 FHD 0.604 0 0.235 0 0 0.006 0 0 0 0.003 0 0 0 0.065
68190 FHD 0.709 0 0.407 0 0 0 0 0 0 0.005 0 0 0 0.086
68189 FHD 0.489 0 0.285 0 0 0.006 0 0.003 0 0.002 0 0 0 0.061
68147 FAV ZČU PiVa 0.451 0 0.151 0 0 0 0 0 0 0.001 0 0 0 0.046
68144 FAV ZČU PiVa 0.287 0 0.093 0 0 0.005 0 0 0 0.004 0 0 0 0.03
68142 FAV ZČU PiVa 0.287 0 0.097 0 0 0 0 0 0 0 0 0 0 0.03
68140 FAV ZČU PiVa 0.501 0 0.227 0 0 0 0 0 0 0.001 0 0 0 0.056
68139 FAV ZČU PiVa 0.23 0 0.113 0 0 0 0 0 0 0.004 0 0 0 0.027
68137 FAV ZČU PiVa 0.507 0 0.185 0 0 0 0 0 0 0.001 0 0 0 0.053
68095 FAV ZČU PiVa 0.308 0 0.094 0 0 0.005 0 0 0 0 0 0 0 0.031
68092 FAV ZČU PiVa 0.543 0 0.251 0 0 0.009 0 0 0 0.001 0 0 0 0.062
67969 FHD 0.685 0 0.337 0 0 0.01 0 0 0 0.002 0 0 0 0.08
67968 FHD 0.718 0 0.33 0 0 0.005 0 0 0 0.007 0 0 0 0.082
67967 FHD 0.602 0 0.286 0 0 0.005 0 0 0 0.002 0 0 0 0.069
67965 FHD 0.345 0 0.297 0 0 0 0 0.002 0 0.014 0 0 0 0.051
67964 FHD 0.561 0 0.387 0 0 0 0 0.002 0 0.002 0 0 0 0.073
67963 FHD 0.409 0 0.365 0 0 0 0 0 0 0.006 0 0 0 0.06
67864 FAV ZČU PiVa 0.33 0 0.098 0 0 0.01 0 0 0 0.004 0 0 0 0.034
67856 FAV ZČU PiVa 0.541 0 0.297 0 0 0.02 0 0 0 0.001 0 0 0 0.066
67620 FAV ZČU CV 0.192 0 0.267 0 0 0 0 0 0 0.002 0 0 0 0.035
Accuracy per substrate: Geographically distinct
Run Group hard_coral_branching hard_coral_submassive hard_coral_boulder hard_coral_encrusting hard_coral_table hard_coral_foliose hard_coral_mushroom soft_coral soft_coral_gorgonian sponge sponge_barrel fire_coral_millepora algae_macro_or_leaves Average
68192 FHD 0.674 0 0.345 0 0 0 0 0 0 0.008 0 0 0 0.079
68191 FHD 0.604 0 0.235 0 0 0.006 0 0 0 0.003 0 0 0 0.065
68190 FHD 0.709 0 0.407 0 0 0 0 0 0 0.005 0 0 0 0.086
68189 FHD 0.489 0 0.285 0 0 0.006 0 0.003 0 0.002 0 0 0 0.061
68147 FAV ZČU PiVa 0.451 0 0.151 0 0 0 0 0 0 0.001 0 0 0 0.046
68144 FAV ZČU PiVa 0.287 0 0.093 0 0 0.005 0 0 0 0.004 0 0 0 0.03
68142 FAV ZČU PiVa 0.287 0 0.097 0 0 0 0 0 0 0 0 0 0 0.03
68140 FAV ZČU PiVa 0.501 0 0.227 0 0 0 0 0 0 0.001 0 0 0 0.056
68139 FAV ZČU PiVa 0.23 0 0.113 0 0 0 0 0 0 0.004 0 0 0 0.027
68137 FAV ZČU PiVa 0.507 0 0.185 0 0 0 0 0 0 0.001 0 0 0 0.053
68095 FAV ZČU PiVa 0.308 0 0.094 0 0 0.005 0 0 0 0 0 0 0 0.031
68092 FAV ZČU PiVa 0.543 0 0.251 0 0 0.009 0 0 0 0.001 0 0 0 0.062
67969 FHD 0.685 0 0.337 0 0 0.01 0 0 0 0.002 0 0 0 0.08
67968 FHD 0.718 0 0.33 0 0 0.005 0 0 0 0.007 0 0 0 0.082
67967 FHD 0.602 0 0.286 0 0 0.005 0 0 0 0.002 0 0 0 0.069
67965 FHD 0.345 0 0.297 0 0 0 0 0.002 0 0.014 0 0 0 0.051
67964 FHD 0.561 0 0.387 0 0 0 0 0.002 0 0.002 0 0 0 0.073
67963 FHD 0.409 0 0.365 0 0 0 0 0 0 0.006 0 0 0 0.06
67864 FAV ZČU PiVa 0.33 0 0.098 0 0 0.01 0 0 0 0.004 0 0 0 0.034
67856 FAV ZČU PiVa 0.541 0 0.297 0 0 0.02 0 0 0 0.001 0 0 0 0.066
67620 FAV ZČU CV 0.192 0 0.267 0 0 0 0 0 0 0.002 0 0 0 0.035

CEUR Working Notes

  • All participating teams with at least one graded submission, regardless of the score, should submit a CEUR working notes paper.
  • The working notes paper should be submitted using this link:
    https://easychair.org/conferences/?conf=clef2020
    and select track "ImageCLEF - Multimedia Retrieval in CLEF".
    Add author information, paper title/abstract, keywords, select "Task 2 - ImageCLEFcoral" and upload your working notes paper as pdf.
  • The working notes are prepared using the LNCS template available at:

    http://www.springer.de/comp/lncs/authors.html

    However, CEUR-WS asks to include the following copyright box in each paper:

    Copyright c 2020 for this paper by its authors. Use permitted under
    Creative Commons License Attribution 4.0 International (CC BY 4.0).
    CLEF 2020, 22-25 September 2020, Thessaloniki, Greece.

    To facilitate authors, we have prepared a LaTex template you can download at:

    https://drive.google.com/file/d/1T-okD-aDIoBHNt1D2CztTcmMGbm80JPG/view?u...

Citations

When referring to the ImageCLEF 2020 coral task general goals, general results, etc. please cite the following publication:

  • Jon Chamberlain, Antonio Campello, Jessica P. Wright, Louis G. Clift, Adrian Clark and Alba García Seco de Herrera. Overview of the ImageCLEFcoral 2020 Task: Automated Coral Reef Image Annotation. CEUR Workshop Proceedings (CEUR- WS.org), http://ceur-ws.org/.
  • BibTex:
    @Inproceedings{ImageCLEFcoraloverview2020,
    author = {Chamberlain, Jon and Campello, Antonio and Wright, Jessica P. and Clift, Louis G. and Clark, Adrian and Garc\'ia Seco de Herrera, Alba},
    title = {Overview of the {ImageCLEFcoral} 2020 Task: Automated Coral Reef Image Annotation},
    booktitle = {CLEF2020 Working Notes},
    series = {{CEUR} Workshop Proceedings},
    year = {2020},
    volume = {1166},
    publisher = {CEUR-WS.org $<$http://ceur-ws.org$>$},
    month = {September 22-25},
    address = {Thessaloniki, Greece}
    }

    • When referring to the ImageCLEF 2020 lab general goals, general results, etc. please cite the following publication (also referred to as ImageCLEF general overview):

    • Bogdan Ionescu, Henning Müller, Renaud Péteri, Asma Ben Abacha, Vivek Datla, Sadid A. Hasan, Dina Demner-Fushman, Serge Kozlovski, Vitali Liauchuk, Yashin Dicente Cid, Vassili Kovalev, Obioma Pelka, Christoph M. Friedrich, Alba García Seco de Herrera, Van-Tu Ninh, Tu-Khiem Le, Liting Zhou, Luca Piras, Michael Riegler, Pål Halvorsen, Minh-Triet Tran, Mathias Lux, Cathal Gurrin, Duc-Tien Dang-Nguyen, Jon Chamberlain, Adrian Clark, Antonio Campello, Dimitri Fichou, Raul Berari, Paul Brie, Mihai Dogariu, Liviu Daniel Ștefan, Mihai Gabriel Constantin, Overview of the ImageCLEF 2020: Overview of the ImageCLEF 2020: Multimedia Retrieval in Medical, Lifelogging, Nature, and Internet Applications In: Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the 11th International Conference of the CLEF Association (CLEF 2020), Thessaloniki, Greece, LNCS Lecture Notes in Computer Science, 12260, Springer (September 22-25, 2020).
    • BibTex:
      @inproceedings{ImageCLEF2020,
        author = {Bogdan Ionescu and Henning M\"uller and Renaud P\'{e}teri and Asma Ben Abacha and Vivek Datla and Sadid A. Hasan and Dina Demner-Fushman and Serge Kozlovski and Vitali Liauchuk and Yashin Dicente Cid and Vassili Kovalev and Obioma Pelka and Christoph M. Friedrich and Alba Garc\'{\i}a Seco de Herrera and Van-Tu Ninh and Tu-Khiem Le and Liting Zhou and Luca Piras and Michael Riegler and P\aa l Halvorsen and Minh-Triet Tran and Mathias Lux and Cathal Gurrin and Duc-Tien Dang-Nguyen and Jon Chamberlain and Adrian Clark and Antonio Campello and Dimitri Fichou and Raul Berari and Paul Brie and Mihai Dogariu and Liviu Daniel \c{S}tefan and Mihai Gabriel Constantin},
        title = {{Overview of the ImageCLEF 2020}: Multimedia Retrieval in Medical, Lifelogging, Nature, and Internet Applications},
        booktitle = {Experimental IR Meets Multilinguality, Multimodality, and Interaction},
        series = {Proceedings of the 11th International Conference of the CLEF Association (CLEF 2020)},
        year = {2020},
        volume = {12260},
        publisher = {{LNCS} Lecture Notes in Computer Science, Springer},
        pages = {},
        month = {September 22-25},
        address = {Thessaloniki, Greece}
        }

        Contact

        Join our mailing list: https://groups.google.com/d/forum/imageclefcoral
        Follow @imageclef

    AttachmentSize
    Image icon imageclefcoral_logo.gif18.04 KB

    ImageCLEF - Cross-language image retrieval evaluations