Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Dualsense device support #645

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

OceanPresentChao
Copy link

What this does

Recently I try to integrate my DualSense controller with robosuite to teleoperate the simulated robot. I finish it and I think maybe we can merge the feature into Robosuite official codebase

Title Label
Dualsense device support (✨ Feature) Added Sony PS5 DualSense device support for teleoperating simulated robot in environment

PS: The doc has not been updated. If Robosuite has a plan to add this feature, I will commit to update.

How it was tested

  • Added dualsense option for device arg in demos/demo_device_control.py.
  • Test with real dualsense controller in MacOS and Unbuntu20.04.

How to checkout & try? (for the reviewer)

Connect your dualsense to your computer through USB or BlueTooth
Run

python -m robosuite.demo.demo_device_control --device dualsense

In MacOS, run

mjpython robosuite/demos/demo_device_control.py --device dualsense

dualsense_demo

SECTION TO REMOVE BEFORE SUBMITTING YOUR PR

Note: Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. Try to avoid tagging more than 3 people.

Note: Before submitting this PR, please read the contributor guideline.

@OceanPresentChao
Copy link
Author

@kevin-thankyou-lin
Copy link
Contributor

Hi @OceanPresentChao, thanks for this awesome contribution! Yes, if you're able to update the docs here that'd be awesome too.

Copy link
Contributor

@kevin-thankyou-lin kevin-thankyou-lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small comment on docs, otherwise looks great and thank you!

| s | switch active arm (if multi-armed robot) |
| = | switch active robot (if multi-robot env) |
| ESC | quit |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ESC still works for quitting across all devices? Maybe let's keep that?

Copy link
Author

@OceanPresentChao OceanPresentChao Mar 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There isn't any 'esc' key listener in source code. I try esc key in both MacOS and Ubuntu but it doesn't work. So I think it is a typographical mistake.
We can add key listener if needed:

elif key == pynput.keyboard.Key.esc:
    print("Detect `esc` key, exiting")
    os._exit(0)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My apologies -- I guess I meant Ctrl + q, as in robosuite.devices.keyboard.py's print_command("Ctrl+q", "reset simulation"). Looks like current docs are incorrect.

I think Ctrl + q is nicer than Ctrl + c because Ctrl + c ends the current program and thus you need to run the python script again to e.g. collect more data using python robosuite/scripts/collect_human_demonstrations.py --device keyboard/dualsense

Could we please update to Ctrl + q? I'll merge after that - thanks for making these updates!

Copy link
Author

@OceanPresentChao OceanPresentChao Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I get what you mean and the difference between quit and reset. The docs docs/modules/devices.md already contain both ctrl + q for reset and ctrl + c for quit. I think we can remain both of them?
image
Actually, it is q instead of ctrl + q in robosuite/devices/keyboard.py. There isn't binding of ctrl and I have tried it : ) .
Of course, ctrl + q still works; it's a very small mistake. Should we update the docs to q?
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see; I think Ctrl in Ctrl+q was deliberately added as a way to escape mujoco viewer logging stuff, from this commit: "update keyboard instruction for reseting: add Ctrl to command to prevent interference with mj viewer".

@kevin-thankyou-lin
Copy link
Contributor

kevin-thankyou-lin commented Mar 3, 2025

Weird, we're getting a failed test on the robot models --- would it be possible you for to run pytest -v tests/test_robots/test_composite_robots.py locally to see if you also get the same error in the test run and paste the output here? Seems like something's wrong w/ the Saywer robot e.g. [None-Sawyer]

@OceanPresentChao
Copy link
Author

Weird, we're getting a failed test on the robot models --- would it be possible you for to run pytest -v tests/test_robots/test_composite_robots.py locally to see if you also get the same error in the test run and paste the output here? Seems like something's wrong w/ the Saywer robot e.g. [None-Sawyer]

Of course. I run python -m pytest locally and get the same errors as the GitHub action. It seems the error of robosuite

This is my MacOS Environment:

Hardware Overview:
      OS Version: MacOS Sequoia 15.3.1
      Model Name: MacBook Air
      Model Identifier: Mac14,2
      Model Number: Z15W0000BZP/A
      Chip: Apple M2
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 16 GB
      System Firmware Version: 11881.81.4
      OS Loader Version: 11881.81.4

Software Overview
    platform darwin 
    Python 3.11.11
    pytest-8.3.4
    pluggy-1.5.0

on latest robosuite master branch

I got errors:

image

image

on current pr dualsense branch

I think it's the same as the master branch

image

image

@kevin-thankyou-lin
Copy link
Contributor

Gotcha, thanks for running this test! I'll debug later this week/weekend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants