IMAGECLEFphoto standard adhoc task submission format ---------------------------------------------------- Introduction ------------- Submission for the ImageCLEF task follows the standard TREC format and the relevant section from the general TREC guidelines have been reproduced almost verbatim below. Note in ImageCLEF, the document reference refers to both the image and caption. The format to use when submitting results is as follows, using a *space* as the delimiter between columns. The width of the columns in the format is not important, but it is important to include all columns and have at least one space between the columns. ... 25 1 00/99 0 4238 xyzT10af5 25 1 00/11 1 4223 xyzT10af5 25 1 16/16998 2 4207 xyzT10af5 25 1 15/15001 3 4194 xyzT10af5 ... etc where: (1) The first column is the topic number -- these will be numbered 1-60 in 2008. (2) The second column is the query number within that topic and these allow for variation between the translations. This field is not used in ImageCLEF 2008 and should be set to 1. (3) The third column is the official document number of the retrieved document. This will take the form of: directory/filename, e.g. "15/15001" where the filename has the extension removed. (4) The fourth column is rank position (**starting from 0**). (5) The fifth column shows the score (integer or floating point) that generated the ranking. This score MUST be in descending (non-increasing) order and is important to include so that we can handle tied scores (for a given run) in a uniform fashion (the evaluation routines rank documents from these scores, not from your ranks). (6) The sixth column is called the "run tag" and should be a unique identifier for your group AND for the method used. That is, each run should have a different tag that identifies the group and the method that produced the run. Submissions ----------- You can submit as many system runs for a given language as you like (ad hoc). We will publish your best result for a given query setting (e.g. automatic vs. manual). Evaluation of system runs will follow the standard TREC methodology. All submissions should be sent by email to Thomas Arni (t.arni@sheffield.ac.uk). Deadlines ---------- The deadline for submissions is 15th June 2008.