Skip to content

Conversation

jiahui-xin
Copy link

This PR introduces changes specifically within the pytorch/ directory to adapt the codebase for newer Gym/Gymnasium API versions.

Problem:
The original code within pytorch/ was incompatible with newer Gym/Gymnasium API versions. This was due to changes in the return values of env.reset() and env.step(), and how states (sometimes tuples) are handled.

Solution:
This PR updates the code within pytorch/ to align with the new Gym API:

  • get_action: Now correctly handles tuple state inputs by extracting the main observation.
  • env.reset(): Updated to state, _ = env.reset() to match the new API (returns observation and info).
  • env.step(): Adapted to process five return values (next_state, reward, terminated, truncated, info) and now uses a combined done = terminated or truncated flag.

These changes ensure the pytorch/ examples/code work correctly with environments using the updated API, such as CartPole.

Testing:

  • Tested the changes in the pytorch/ folder with: gym 0.26.2, Python 3.9.13 on macOS 15.4.1 (Sequoia).

- Handles tuple states in get_action.

- Adapts env.reset() and env.step() for new API signatures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant