Documenting advanced functionality

[nschneid]

Moving from #3:

Exo (exercise) modes:

• teacher visible mode: dumb exercise where students have to copy the teacher's tree which is visible but not directly modifiable. a good start with 3 sentences or so.
• no feedback: the student can't see the teacher's tree and gets no feedback, but the admin can export the results of the students' annotations compared to the teacher trees
• percentage: when students save, they can see how many percent they got wrong of dependencies and pos, but they don't know where
• graphical feedback: when students save, they can see where there are problems compared to the teacher's tree and they have to find the right annotation.

baseAnnotatorName = parser

baseAnnotatorName is no longer used. to make it clearer that this is important only for exercise modes the variable is called exoBaseAnnotatorName. it is the name of the user that provides the tree that the student should start with.
teacher is used for comparisons and results in exercises.

I assigned the entire sample to the user. Is there a way to assign only particular sentences?

yes, but not from the graphical interface. there's code to intelligently distribute new sentences mixed up with already annotated sentences in a way that two students don't have the same set of sentences to annotate. this is used to class-source dependency annotation...

[nschneid]

Cool, I was wondering how to upload a dataset with gold standard annotations hidden from annotators so that I can measure their accuracy. The way to do that is to upload the dataset and then set teacher to be the same value as importAnnotatorName?

[kimgerdes]

yes. and choose the exercise mode "no feedback"

[kimgerdes]

did this work as expected?

[MagaliDuran]

We are using the Arborator-Grew, exercise mode, to train annotators. The facilities of the tool are excellent!
However, we had difficulty deducing that the exercise levels (1 to 4) available in the tool correspond to the exercise modes (teacher visible mode, no feedback, percentage, graphical feedback) described in the documentation. So, we kindly asked you make this information clear on the project wiki page: https://github.com/Arborator/arborator-server/wiki