Practical showcase of medical image retrieval systems

Medical User-oriented Image Retrieval Task

This track is designed to foster research in Interactive Information retrieval and user seeking behavior of medical images. Aspects that could be explored include the users' needs and behaviors when they search for medical images as well as comparison of different interfaces. Groups that are interested in testing the human computer interaction aspects of the image retrieval problem as well as researchers interested in exploring issues related to user seeking behavior are encouraged to participate.

For this year we will run this as a demo session during the CLEF 2011 conference.

Data Collection

We will be using the same collection used in the 2011 Ad-hoc image based task.

Evaluation measures

For this task we propose to use the following:

Effectiveness Measures:
• Interactive recall: Number of CLEF-2010 relevant images saved by the user/number of CLEF-2011 relevant images in the corpus
• Interactive user precision: Number of CLEF-2010 relevant images saved by the user/total number of images saved by the user
• Mean Reciprocal rank (computed on the top 10 or top 20 results) and based on the user’s saved images and or the CLEF-2011 saved images.

Efficiency:
• Search Speed: The proportion of relevant images that are found per minute. Number of saved images/ total number of minutes spend to complete the topic.

Satisfaction:
• Satisfaction with results retrieved by the system: Measured as a 10 points Likert Scale.

Overall usability measures:
• Overall satisfaction with results: Measured as a 10-point scale.
• Satisfaction with the speed of response: Measured as a 10-point scale.
• Ease of use: Measured as a 10-point scale.
• Ease of learning: Measured as a 10-point scale.
• Perceived usefulness: Measured as a 10-point scale.

Contact

Please contact Miguel E Ruiz (meruiz@gmail.com) for any questions about this task.