diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..855c94c --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,53 @@ +# CHANGELOG + +## [Unreleased] - yyyy-mm-dd + +### Added + +### Changed + +### Fixed + + +## [[1.1.0] - 2021-10-13](https://github.com/AndrejOrsula/drl_grasping/releases/tag/1.1.0) + +### Added + +- `kinova_j2s7s300` is now added to the supported robot models. +- Custom SDF worlds are now used as the base for RL environments. +- Support for `DRL_GRASPING_DEBUG_LEVEL` environment variable. +- Ignition fortress is tested and fully functional. +- ROS 2 rolling is tested and fully functional. + +### Changed + +- Local implementation of conversions for quaternion sequence is now used. +- Simplified installation instructions in README.md. +- Simplified and improved Dockerfile. + +### Fixed + +- Compatibility with Stable-Baselines3 v1.2.0 + + +## [[1.0.0] - 2021-06-08](https://github.com/AndrejOrsula/drl_grasping/releases/tag/1.0.0) + +### Added + +- Initial version of this project. + - Supported environments + - `Reach-Gazebo-v0` + - `Reach-ColorImage-Gazebo-v0` + - `Reach-Octree-Gazebo-v0` + - `Reach-OctreeWithColor-Gazebo-v0` + - `Grasp-Octree-Gazebo-v0` + - `Grasp-OctreeWithColor-Gazebo-v0` + - Supported robot models + - `panda` + - `ur5_rg2` + - Custom feature extractor + - `OctreeCnnFeaturesExtractor` + - Tested RL algorithms + - `td3` + - `sac` + - `tqc` diff --git a/README.md b/README.md index d1aff52..325e490 100644 --- a/README.md +++ b/README.md @@ -30,27 +30,31 @@ Example of Sim2Real transfer on UR5 can be seen below (trained inside simulation ### Dependencies -These dependencies are required to use the entirety of this project. If no "(tested with `version`)" is specified, the latest release from a relevant distribution is expected to function properly. - -- [Python 3](https://www.python.org/downloads) (tested with `3.8`) -- [PyTorch](https://github.com/pytorch/pytorch#installation) (tested with `1.7`) -- [ROS 2 Foxy](https://index.ros.org/doc/ros2/Installation/Foxy) -- [Ignition Dome](https://ignitionrobotics.org/docs/dome/install) -- [MoveIt 2](https://moveit.ros.org/install-moveit2/source) +These are the primary dependencies required to use this project. + +- [Python 3.8](https://www.python.org/downloads) +- ROS 2 [Foxy](https://docs.ros.org/en/foxy/Installation.html) OR [Rolling (recommended)](https://docs.ros.org/en/rolling/Installation.html) +- Ignition [Dome](https://ignitionrobotics.org/docs/dome/install) OR [Fortress (recommended)](https://ignitionrobotics.org/docs/fortress) +- [MoveIt 2](https://moveit.ros.org/install-moveit2/binary) + - Install/build a version based on the selected ROS 2 release +- [ros_ign](https://github.com/ignitionrobotics/ros_ign/tree/ros2) + - Install/build a version based on the selected combination of ROS 2 release and Ignition version - [gym-ignition](https://github.com/robotology/gym-ignition) - [AndrejOrsula/gym-ignition](https://github.com/AndrejOrsula/gym-ignition) fork is currently required - [O-CNN](https://github.com/microsoft/O-CNN) - [AndrejOrsula/O-CNN](https://github.com/AndrejOrsula/O-CNN) fork is currently required +- [PyTorch](https://pytorch.org/get-started/locally) (last tested on 1.9.1) +- [Stable-Baselines3](https://stable-baselines3.readthedocs.io/en/master/guide/install.html) (last tested on 1.2.0) and [sb3-contrib](https://stable-baselines3.readthedocs.io/en/master/guide/sb3_contrib.html#installation) -Several other dependencies can be installed via `pip` with this one-liner. +Python dependencies are listed under [python_requirements.txt](./python_requirements.txt). All of these (including Pytorch and Stable-Baselines3) can be installed via `pip`. ```bash -pip3 install numpy scipy optuna seaborn stable-baselines3[extra] sb3-contrib open3d trimesh pcg-gazebo +pip3 install -r python_requirements.txt ``` -All other dependencies are pulled from git and built together with this repository, see [drl_grasping.repos](drl_grasping.repos) for more details. +Dependencies for robot models (e.g. [panda_ign](https://github.com/AndrejOrsula/panda_ign)/[panda_moveit2_config](https://github.com/AndrejOrsula/panda_moveit2_config)) and interaction between MoveIt 2 and Ignition ([ign_moveit2](https://github.com/AndrejOrsula/ign_moveit2)) are pulled from git and built together with this repository, see [drl_grasping.repos](drl_grasping.repos) for more details. -> In case you run into any problems with dependencies along the way, check [Dockerfile](docker/Dockerfile) that includes the full instructions. +> In case you run into any problems with dependencies along the way, please check [Dockerfile](docker/Dockerfile) that includes the full instructions. ### Building @@ -63,7 +67,7 @@ mkdir -p drl_grasping/src && cd drl_grasping/src git clone https://github.com/AndrejOrsula/drl_grasping.git # Import and install dependencies vcs import < drl_grasping/drl_grasping.repos && cd .. -rosdep install -r --from-paths src -i -y --rosdistro ${ROS_DISTRO} +rosdep install -r --from-paths src -i --rosdistro ${ROS_DISTRO} # Build with colcon colcon build --merge-install --symlink-install --cmake-args "-DCMAKE_BUILD_TYPE=Release" ``` @@ -98,7 +102,7 @@ sudo systemctl restart docker ### Pre-built Docker Image -The easiest way to try out this project is by using a pre-built Docker image that can be pulled from [Docker Hub](https://hub.docker.com/repository/docker/andrejorsula/drl_grasping). Currently, there is only a development image available (large, but allows editing and recompiling), which also contains the default testing datasets for ease of use. You can pull the latest tag with the following command (~7.5 GB with all parent images). +The easiest way to try out this project is by using a pre-built Docker image that can be pulled from [Docker Hub](https://hub.docker.com/repository/docker/andrejorsula/drl_grasping). Currently, there is only a development image available that also contains the default testing datasets (huge, but it is easy to use and allows editing and recompiling). You can pull the `latest` tag with the following command. Alternatively, each release has also its associated tag, e.g. `1.0.0`. ```bash docker pull andrejorsula/drl_grasping:latest @@ -116,7 +120,7 @@ If desired, you can also run examples and scripts directly with this setup, e.g. /docker/run.bash andrejorsula/drl_grasping:latest ros2 run drl_grasping ex_enjoy_pretrained_agent.bash ``` -> If you are struggling to get CUDA working on your system with Nvidia GPU (no `nvidia-smi` output), you might need to use a different version of CUDA base image that supports the version of your driver. +> If you are struggling to get CUDA working on your system with Nvidia GPU (no `nvidia-smi` output), you might need to use a different version of CUDA base image that supports the version of your driver. If that is the case, you need to build yourself a new Docker image. ### Building a New Image @@ -139,6 +143,10 @@ This enables: - Execution of scripts and examples via `ros2 run drl_grasping ` - Launching of setup scripts via `ros2 launch drl_grasping ` +### Debug Level + +Environment variable `DRL_GRASPING_DEBUG_LEVEL` can be set `DEBUG`/`INFO`/`WARN`/`ERROR`/`DISABLED` to affect the level of logging for environments. +
Using Pre-trained Agents (click to expand) @@ -161,7 +169,7 @@ The default agent is for `Grasp-OctreeWithColor-Gazebo-v0` environment with Pand ### Training of Agent -To train your own agent, you can start with the [`ex_train.bash`](examples/ex_train.bash) example. You can customise this example script, configuration of the environment and all hyperparameters to your needs (see below). By default, headless mode is used during training to reduce computational load. If you want to see what is going on, use `ign gazebo -g` or `ROS_DOMAIN_ID=69 rviz2` and visualise point cloud of the scene. +To train your own agent, you can start with the [`ex_train.bash`](examples/ex_train.bash) example. You can customise this example script, configuration of the environment and all hyperparameters to your needs (see below). By default, headless mode is used during training to reduce computational load. If you want to see what is going on, use `ign gazebo -g` or `ROS_DOMAIN_ID=69 rviz2` (`ROS_DOMAIN_ID=69` is default for Docker image). ```bash ros2 run drl_grasping ex_train.bash @@ -259,7 +267,7 @@ There are several databases with free PBR textures that you can use. Alternative ### Supported Robots -Only [Franka Emika Panda](https://github.com/AndrejOrsula/panda_ign) and [UR5 with RG2 gripper](https://github.com/AndrejOrsula/ur5_rg2_ign) are supported. This project currently lacks a more generic solution that would allow to easily utilize arbitrary models, e.g. full-on [MoveIt 2](https://github.com/ros-planning/moveit2) with [ros2_control](https://github.com/ros-controls/ros2_control) implementation. Adding new models is not complicated though, just time-consuming. +Only [Franka Emika Panda](https://github.com/AndrejOrsula/panda_ign), [UR5 with RG2 gripper](https://github.com/AndrejOrsula/ur5_rg2_ign) and [Kinova Gen2 (j2s7s300)](https://github.com/AndrejOrsula/kinova_j2s7s300_ign) are supported. This project currently lacks a more generic solution that would allow to easily utilize arbitrary models, e.g. full-on [MoveIt 2](https://github.com/ros-planning/moveit2) with [ros2_control](https://github.com/ros-controls/ros2_control) implementation. Adding new models is not complicated though, just time-consuming. ## Reinforcement Learning diff --git a/drl_grasping/envs/randomizers/manipulation.py b/drl_grasping/envs/randomizers/manipulation.py index e8ccb2d..daa8677 100644 --- a/drl_grasping/envs/randomizers/manipulation.py +++ b/drl_grasping/envs/randomizers/manipulation.py @@ -587,6 +587,7 @@ def randomize_object_models(self, volume=task._object_spawn_volume, np_random=task.np_random) try: + model_name = '' model = models.RandomObject(world=task.world, position=position, orientation=quat_random, @@ -598,9 +599,9 @@ def randomize_object_models(self, link = model.to_gazebo().get_link( link_name=model.link_names()[0]) link.enable_contact_detection(True) - except: - # TODO (low priority): Automatically blacklist a model if Gazebo does not accept it - pass + except Exception as ex: + print("Model " + model_name + + " could not be insterted. Reason: " + str(ex)) def randomize_object_primitives(self, task: SupportedTasks): diff --git a/drl_grasping/envs/tasks/__init__.py b/drl_grasping/envs/tasks/__init__.py index 8b533f6..1378da3 100644 --- a/drl_grasping/envs/tasks/__init__.py +++ b/drl_grasping/envs/tasks/__init__.py @@ -1,10 +1,20 @@ +from gym import logger as gym_logger +from gym_ignition.utils import logger as gym_ign_logger from gym.envs.registration import register +from ament_index_python.packages import get_package_share_directory +from os import path, environ from .reach import Reach, ReachColorImage, ReachDepthImage, ReachOctree from .grasp import Grasp, GraspOctree -from ament_index_python.packages import get_package_share_directory -from os import path +# Set debug level +debug_level = environ.get('DRL_GRASPING_DEBUG_LEVEL', default='ERROR') +gym_ign_logger.set_level( + level=getattr(gym_logger, debug_level), + scenario_level=getattr(gym_logger, debug_level) +) + +# Get path for worlds worlds_dir = path.join(get_package_share_directory('drl_grasping'), 'worlds') # Reach diff --git a/drl_grasping/envs/worlds/default.sdf b/drl_grasping/envs/worlds/default.sdf index 0639866..7e691df 100644 --- a/drl_grasping/envs/worlds/default.sdf +++ b/drl_grasping/envs/worlds/default.sdf @@ -80,7 +80,7 @@ - 250 + 275 110 true true diff --git a/drl_grasping/envs/worlds/default_interactive.sdf b/drl_grasping/envs/worlds/default_interactive.sdf index ebcc28b..0b8c8df 100644 --- a/drl_grasping/envs/worlds/default_interactive.sdf +++ b/drl_grasping/envs/worlds/default_interactive.sdf @@ -87,7 +87,7 @@ - 250 + 275 110 true true diff --git a/python_requirements.txt b/python_requirements.txt new file mode 120000 index 0000000..737ae56 --- /dev/null +++ b/python_requirements.txt @@ -0,0 +1 @@ +docker/python_requirements.txt \ No newline at end of file