Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ALE 0.6 - modes and difficulties for Atari games #1535

Merged
merged 1 commit into from
Jun 21, 2019
Merged

ALE 0.6 - modes and difficulties for Atari games #1535

merged 1 commit into from
Jun 21, 2019

Conversation

JesseFarebro
Copy link
Contributor

@JesseFarebro JesseFarebro commented Jun 15, 2019

This PR along with openai/atari-py#49 introduces modes and difficulties for Atari 2600 games in Gym. This was introduced in version 0.6 of the ALE and also outlined in the paper "Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents" by Machado et al.

I had previously discussed versioning in openai/atari-py#49 with @christopherhesse as this PR would rely on a new API in v0.6 which allows you to query and set the mode and/or difficulty.

The other thing to note in this PR is the removal of self.game_path. I don't understand why you call ale.loadROM in the seed function. To my knowledge, you don't need to set the random seed before loading the ROM. The seed only affects the RNG for sticky actions. In fact, most examples in the ALE repository set the seed after loading the ROM.
I had to make this change as it seems seed gets called twice and the second call would override any previously defined modes or difficulties (each call to loadROM resets the mode and difficulty to the default of 0).

To get this PR merged I think we should discuss:

  1. How to handle versioning in Gym with respect to atari-py. It's probably necessary to check for setDifficulty and setMode in atari-py. If the newer API isn't found we can throw an error asking the user to upgrade atari-py.
  2. Gain some clarity on the ale.loadRom call in seed. I don't see this breaking anything but it would be nice to get some clarity on why this existed in the first place.

@christopherhesse
Copy link
Contributor

For 1), why not just require atari-py~=0.2.0 in gym?

@JesseFarebro
Copy link
Contributor Author

@christopherhesse I agree, I'm not too familiar with pip's upgrade strategy but if we specify atari-py~=0.2.0 in gym will it indeed upgrade atari-py even though the major version hasn't changed?

We'll have to bump atari-py then I can reflect that change in this PR.

@christopherhesse
Copy link
Contributor

Yeah it should work, the major version for a pre 1.0 release should be the second digit. For that requirement https://www.python.org/dev/peps/pep-0440/#compatible-release indicates that it would require atari-py>=0.2.0, atari-py==0.2.*.

@JesseFarebro
Copy link
Contributor Author

Removing loadROM from seed ends up breaking determinism (I can see how since the ALE manages Stella's RNG after a loadROM call). Without looking too much more into it I just figured I'd leave the loadROM call in seed. This way I just set the mode/difficulty in seed.

I have tested everything and it's working as I expect. This is ready for someone to take a look at now.

@JesseFarebro JesseFarebro marked this pull request as ready for review June 20, 2019 19:03
@pzhokhov pzhokhov merged commit fc6891c into openai:master Jun 21, 2019
@pzhokhov
Copy link
Collaborator

LGTM, let's merge this, I'll bump version and release the new pypi package.

@JesseFarebro
Copy link
Contributor Author

Thanks for everything @pzhokhov and @christopherhesse!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants