Procedures
There are several datasets included in the challenge. Some of them
are real transcribed medieval manuscripts, some artificial (but
handcopied) ones. The datasets are made available in (at least) two
phases. Correct answers to some of the first phase datasets will be
announced before the final submission deadline in order to enable
selfassessment by the participants. These datasets will not affect
the final results.
Submissions are made in groups (of size one or more persons). Each
group should submit at most two solutions. If more than two solutions
are submitted, the last two before the deadline are accepted. A person
can belong to at most two groups. (So a person can contribute to a
maximum of four solutions.)
Evaluation
For the artificial datasets, for which there is a known
'groundtruth' solution, the difference between the proposed and
correct solutions is evaluation. The exact criterion is open for
discussion among participants (and other interested parties). The
current scoring function is based on distance orders among triplets
(for details, see Example) Go to Discussion.
For the real datasets, there is no known correct solution. Therefore,
EBS¹ will be used: the 'owner' of the dataset will be asked to
estimate whether the proposed solution is plausible and useful from
his/her point of view.
¹ EBS: endorphinbased scoring
Ranking Schemes
There are two ranking schemes.
 Primary ranking is based on
performance on the primary dataset (see Data Sets).
 Secondary
ranking is based on all the other data sets except those for which
the correct solution is annouced during the challenge, including both
artificial and real data. For these other data sets, 'thumbsup' marks
will be awarded to the best submissions, and the total number of
thumbs up determines the secondary ranking.
The organizers reserve the right to alter the rules of the challenge
at any point.
Restrictions
Anyone who has been in contact with some of the datasets, or has
otherwise obtained inside information about the correct solution
should obviously not enter the challenge as regards the dataset(s) in
question. However, participation is allowed as regards the other
datasets.
The datasets are provided only for the use in the challenge.
Further use of the data requires an explicit permission by the
organizers and the original providers of the data.
