SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting
This repository contains the code for the paper "SplatSim".
conda env create -f environment.yml
conda activate splatsim
pip3 install -r requirements.txt
cd gello
pip install -e .
cd pybullet-URDF-models
pip install -e .
render_fk_all_highres.py -s /path/to/test_data/robot_iphone -m /path/to/output/robot_iphone --objects plastic_apple --traj_folder /path/to/bc_data/gello
This file contains the code for rendering forward kinematics of the robot as well as the objects in the scene.
2. The folder "gello" contains the code for trajectory generation, which is then rendered using the code in the first file.
Currently, there is a lot of code here, need to clean it up and only keep files that are necessary.
Very unorganized, will need to clean it up.
-
Proper instructions for installation. Current instructions might not work.- Installation instructions should work now.
-
Add links to pretrain gaussian-splats and trajectories, so that people can run rendering script.- Links to pretrain gaussian-splats, colmap and trajectories are added.
- Create a new file for rendering robot and objects, without hardcoding the segmentation and shifting everything to KNN based segmentation.
- Clean up the splat folder for only keeping necessary files and easy creation of KNN based segmentation for robots.
- Documentation for the codebase.
- Adding new robots (in sim or any other environment).
- Instructions to generate a trajectory and render it.
- Trajectory format should be specified properly.
- Clean up the gello folder and only keep files that are necessary.