Skip to content

Official release of the CLVR Jaco Play Dataset, Dass et al. 2023

License

Notifications You must be signed in to change notification settings

clvrai/clvr_jaco_play_dataset

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CLVR Jaco Play Dataset

Open In Colab

Shivin Dass1, Jullian Yapeter1, Jesse Zhang1, Jiahui Zhang1, Karl Pertsch1, Stefanos Nikolaidis1, Joseph J. Lim1,2

1University of Southern California 2KAIST

The CLVR Jaco Play Dataset consists of 1085 teleoperated robot episodes, including language annotations. Please find details for loading and using the data below and in the provided Colab.

For further questions, please contact {sdass, pertsch}@usc.edu.

Downloading the Data

Please install gdown (pip install gdown) and run the provided download script.

bash download_jaco_data.sh ./data

Alternatively, you can manually download the data from this link (~16GB). We found manual download to be faster in some cases.

Using the Data

Please check out our Colab for an example how to unpack and use the data!

The dataset was collected at a frequency of 10Hz. It has the following structure:

  • Observations are split into 5 attributes,
    • front_cam_ob : observations from 3rd person cam
    • mount_cam_ob : observations from mounted camera
    • ee_cartesian_pos_ob : end effector cartesian position. ee_cartesian_pos_ob[0:3] corresponds to position and ee_cartesian_pos_ob[3:7] corresponds to orientation in quarternian format
    • ee_cartesian_vel_ob : end effector cartesian velocity. ee_cartesian_pos_ob[0:3] corresponds to change in position and ee_cartesian_pos_ob[3:6] corresponds to change in orientation in roll, pitch yaw format
    • joint_pos_ob : joint positions of the jaco arm (we only use the last 2 elements of this that correspond to the gripper joints)
  • actions : first 3 elements are cartesian deltas and 4th element is a label from {0, 1, 2} meaning {open gripper, don't move gripper, close gripper}
  • terminals : 1 at the end of each skill
  • prompts : Natural language description of the goal
  • reward : 1 at the end of each skill

Citation

If you find this work useful in your research, please consider citing:

@software{dass2023jacoplay,
  author = {Dass, Shivin and Yapeter, Jullian and Zhang, Jesse and Zhang, Jiahui
            and Pertsch, Karl and Nikolaidis, Stefanos and Lim, Joseph J.},
  title = {CLVR Jaco Play Dataset},
  url = {https://github.com/clvrai/clvr_jaco_play_dataset},
  version = {1.0.0},
  year = {2023}
}

License

The data is available under the Creative Commons Attribution 4.0 International License.

About

Official release of the CLVR Jaco Play Dataset, Dass et al. 2023

Resources

License

Stars

Watchers

Forks

Languages