Important: Updated Rules and Deadline Extension

[luator]

We have been made aware that the order of the episodes in the "mixed" datasets gives away which episodes are recorded with a good policy. It is therefore straightforward to drop all other episodes, which leads to performance gains with behavioral cloning like algorithms. This kind of filtering is not intended as the goal of the task is to see how well the training algorithms can work with datasets of mixed quality.

We thus added a new rule that it is not allowed to filter the dataset based on the position of a sample in the dataset. You may still filter based on the properties of a transition or an episode if you want to.

What does this mean in practise?

• If you do not make explicit use of the order of samples in the dataset, you don't need to change anything.
• If you do currently use the order, e.g. for filtering, please remove that part from your training routine and train a new policy.

We apologise for the inconvenience this may cause, especially so short before the deadline. We therefore extend the deadline until October 7th, to give you some additional time to adapt to the new rule.

The Challenge Organisers