Check list for various use cases of the repo

[riley-mld]

This is a check list of what would need to be done for various use cases of the repo. This list doesn't include what we would need for using Emote in production for our projects.

Basics:

• Better logging #18
• Missing standard/basic training features #27
• N-Step #19
• NANs in high learning rates #20

Open-source:

• Gumbel soft-max for discrete action spaces #21
• More documentation #22

Research for Arkadia:

• PPO (This includes modernising PPO with improvements like GAE) #23
• GAIL #24
• Connect to Emit #25
• GPU only training with Emit and Emote #26

Nice to haves:

• More advanced training/architecture stuff (Spectral normalization, Observation normalization, Reward normalization, Q-Value/Advantage normalization, ...)
• More network types maybe?
• Support for Multi-Discrete action spaces
• CURL or other vision methods
• Hyper parameter tuning

[singhblom]

Could we maybe make concrete individual issues with the steps mentioned here that are the highest priorities? Some of these I don't think are high priorities, some would need thinking and maybe some discussion to make sure the abstractions are good, etc. Having them as separate issues would make sure the discussion ends up in the right place.

[riley-mld]

Sure that sounds like a good idea! I just made issues for the ones that I think that have the highest priority for now. I've added a small description for some of the issues and I'll try to add more details and add a description for the rest as well.