Skip to content

[ICRA, 2025] SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting

License

GPL-2.0, Unknown licenses found

Licenses found

GPL-2.0
LICENSE
Unknown
LICENSE.md
Notifications You must be signed in to change notification settings

qureshinomaan/SplatSim

Folders and files

NameName
Last commit message
Last commit date

Latest commit

d537ec2 · Feb 7, 2025

History

17 Commits
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Feb 4, 2025
Feb 3, 2025
Nov 23, 2024
Feb 7, 2025
Nov 23, 2024
Nov 23, 2024
Feb 7, 2025
Nov 23, 2024
Nov 23, 2024
Oct 8, 2024
Nov 23, 2024
Feb 6, 2025
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Jan 29, 2025
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Nov 23, 2024
Jan 30, 2025
Feb 7, 2025
Feb 3, 2025
Feb 6, 2025
Nov 23, 2024
Nov 23, 2024

Repository files navigation

Readme under construction

SplatSim

SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting

Project Page | Arxiv

This repository contains the code for the paper "SplatSim".

Installation

conda env create -f environment.yml
conda activate splatsim
pip3 install -r requirements.txt
cd gello
pip install -e .
cd pybullet-URDF-models
pip install -e .

Running the rendering code

1. Download the colmap and gaussian-splatting models from the below links:

2. Run the rendering script:

render_fk_all_highres.py -s /path/to/test_data/robot_iphone -m /path/to/output/robot_iphone --objects plastic_apple --traj_folder /path/to/bc_data/gello

A general description of the codebase

1. The most important file is render_fk_all_highres.py

This file contains the code for rendering forward kinematics of the robot as well as the objects in the scene.

2. The folder "gello" contains the code for trajectory generation, which is then rendered using the code in the first file.

Currently, there is a lot of code here, need to clean it up and only keep files that are necessary.

3. The folder "splat" contains the code for KNN logic and some other utilities.

Very unorganized, will need to clean it up.

A list of TODOs

  • Proper instructions for installation. Current instructions might not work.
    • Installation instructions should work now.
  • Add links to pretrain gaussian-splats and trajectories, so that people can run rendering script.
    • Links to pretrain gaussian-splats, colmap and trajectories are added.
  • Create a new file for rendering robot and objects, without hardcoding the segmentation and shifting everything to KNN based segmentation.
  • Clean up the splat folder for only keeping necessary files and easy creation of KNN based segmentation for robots.
  • Documentation for the codebase.
  • Adding new robots (in sim or any other environment).
  • Instructions to generate a trajectory and render it.
    • Trajectory format should be specified properly.
  • Clean up the gello folder and only keep files that are necessary.

About

[ICRA, 2025] SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting

Resources

License

GPL-2.0, Unknown licenses found

Licenses found

GPL-2.0
LICENSE
Unknown
LICENSE.md

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published