Setuptools error

[MattStammers]

I have been making a lot of RL models using sample-factory. Then I stumbled across this very nice RL implementation by @sgoodfriend.

He has set this up really well. It only works in a Linux / Mac environment ub

One thing that might well trip people up is that the setup requires setuptools==65.5.0. This bug is well-documented already on github and can be solved by following: Setuptools error with gym or Gym 0.21 installation issue

The second error I encountered was when installing gym==0.21.0. This error is documented here and is literally caused by a small typo in the setup.py file. Gym Setup.py issue. It can be fixed by installing this version of gym==0.21.0:

pip install git+https://github.com/openai/gym.git@9180d12e1b66e7e2a1a622614f787a6ec147ac40

After I ran this setup.sh worked fine and I was able to run the rest. Good work Scott!

[sgoodfriend]

Thanks for working through this!

I had to fight through installation issues caused by the old version of gym (soon I'll be upgrading to gymnasium!).

https://github.com/sgoodfriend/rl-algo-impls/blob/main/pyproject.toml#L101 is supposed to require the build system to be using the correct old versions of setuptools and wheel to get through installation, but I know it doesn't always seem to work. I've resorted to instructing the installation of the correct versions of setuptools and wheel beforehand.

The installing a specific commit of gym is new to me, though.

To reproduce on my end, can you give information on your dev environment:

• OS
• Python, pip, setuptools, wheel versions
• local environment or some cloud provider?

[MattStammers]

No worries @sgoodfriend. I was running:

1. Jammy-Jellyfish (Ubuntu 22.04)
2. Python 3.10
3. Pip was 23 but downgraded to 21 just in case this was also causing issues
4. Wheel was the same as yours - I didn't change it. I believe >=0.40 has similar issues.
5. Setuptools was originally 68. It was not until I changed this to 65.5.0 before trying to install the package that it worked.
6. Local environment (bare-metal server - HPE Proliant Gen 10 - ML380).

I also had a second bug related to gym==0.21.0 that I didn't discover until later on. It seems that some versions of gym==0.21.0 have an issue with seeds. Because I was in a hurry I tried bumping the version to 0.22.0 after setting everything up and it overcame the problem (presumably this issue was patched in 0.22.0). I have probably broken something else by doing this but have had no issue with the basic environments - either Atari or Pybullet environments. The benchmarking as you said takes over 24 hours to run but it's now at least 20 hours in (running on a P2200 which is less powerful than an A10) and it has reached Qbert so far without breaking down.

Anyway really like the way you have set things up and thanks again for taking the time to open-source it. Has saved me bags of time!

[MattStammers]

The PPO run is now completed but there is still a long way to go so I can see my measly P2200 is nowhere near as grunty as the A10 would be - at the same time it didn't crash which is a good start. I'll see what happens when I try microRTS or something else after I've finished my sample factory experiment.

[MattStammers]

Seems to all work mate. The main feature I would like to add would be the parallelization of policy training across multiple GPU's but still getting my head around how this is implemented in sample factory. Very good work and as I said I am very grateful because you have blazed the trail and helped me enormously.

[sgoodfriend]

@MattStammers , I've upgraded this to use poetry and gymnasium. I'm closing this issue because your reported issue shouldn't be a problem anymore.