A first bare bones paralleled implementation of Go Explore as described by the Uber Engineering blog post
Currently no deep learning is incorperated with the project. The avalible exploration policies are random, and markov chain.
The notebook syncGoExplore.ipynb demonstrates the use of Go Explore to create a speedrun of level in a gym environment using multiple threads.
Dependencies:
ray (linux and osx only)
gym retro
imageio (also needs freeimage)
rom file for the game environment
Original reddit discussion with some more information: https://www.reddit.com/r/MachineLearning/comments/agf43s/d_go_explore_vs_sonic_the_hedgehog/
Original blog post by Uber: https://eng.uber.com/go-explore/
To do:
Add smarter exploration policies (fast simple models and deep learning)
Asynchronous Go Explore, i.e. allow workers to be constantly playing and updating only when ready/neccesary
Add iterative deepening
Add procedures for experiments to search for good hyperparameters
Add the comb operation - sequentially go to each state encountered in a run that reaches the end of the level
Some early gameplay:
A very polished run:
