gradio demo

README.md

@@ -2,7 +2,7 @@
 
 # Differentiable Robot Rendering
 
-[Project Page](https://drrobot.cs.columbia.edu/) | [Video](https://drrobot.cs.columbia.edu/assets/videos/video.mp4) | [Arxiv ]([https://arxiv.org/abs/2410.13851])
+[Project Page](https://drrobot.cs.columbia.edu/) | [Video](https://drrobot.cs.columbia.edu/assets/videos/video.mp4) | [Arxiv](https://arxiv.org/abs/2410.13851)
 
 </div>
 
@@ -19,7 +19,7 @@ This is the official repository for Differentiable Robot Rendering. It includes
 Much of this codebase is inherited from the [original 3D Splatting](https://github.com/graphdeco-inria/gaussian-splatting) codebase, which we simplified as much as possible.  -->
 
 
-## Installation instructions
+## Setup 🛠️
 
 Our setup has been tested with [miniforge](https://github.com/conda-forge/miniforge) and CUDA 12.1. To install all of our dependencies, simply run
 
@@ -32,12 +32,12 @@ pip install tensorboard ray tqdm mujoco open3d plyfile pytorch-kinematics random
 ```
 
 The most tricky dependency of our codebase is [gsplat](https://github.com/nerfstudio-project/gsplat), which is used for rasterizing Gaussians. We recommend visiting their installation instructions if the plain `pip install` doesn't work. 
+
+# Real-Time Robot Reconstruction Demo 😎
 
-# Real-Time Robot Reconstruction Demo
-
-To launch the real-time reconstruction demo, you may use the pre-trained robot model that we include in this repo by simply running
-```
-python gradio_app_realtime.py --model_path ./ur5
+To launch the real-time reconstruction demo, you may use the pre-trained UR5 robot model that we include in this repo by simply running
+```bash
+python gradio_app_realtime.py --model_path assets/ur
 ```
 
 <div align="center">
@@ -46,7 +46,7 @@ python gradio_app_realtime.py --model_path ./ur5
 
 In this demo, you may tweak with MuJoCo robot parameters through the sliders, and the 3D Gaussian robot will be optimized in real-time to reconstruct the ground-truth robot pose. 
 
-# Training
+# Training 🚂
 
 The training is divided into two phases
 1. **Data generation**, which takes in a URDF file and generates randomly sampled combinations of images + camera poses + robot joint angles.
@@ -57,8 +57,8 @@ The training is divided into two phases
      <li>Joint training</li>
    </ol>
 
-## 1. Data Generation 
-
+## 1. Data Generation 💽
+ 
 Our code generates data from a URDF specified as an XML file. We provide several standard robot URDFs from [Mujoco Menagerie](https://github.com/google-deepmind/mujoco_menagerie) to train on, which you can check out under `mujoco_menagerie/`. As an example, we will be using the UR5e robot arm. To generate data for this robot, simply run
 ```
 python generate_robot_data.py --model_xml_dir mujoco_menagerie/universal_robots_ur5e
@@ -90,7 +90,7 @@ data/
 
 P.S. this script is mostly standalone and quite hackable, so you may customize it to your needs.
 
-## 2. Differentiable Robot Training
+## 2. Differentiable Robot Training 👨‍🍳
 
 To train a differentiable robot model, run:
 
@@ -101,3 +101,16 @@ python train.py --dataset_path data/universal_robots_ur5e --experiment_name univ
 This script will automatically run through all three stages of training. The latest robot model will be saved under `output/universal_robots_ur5e/`.
 
 To visualize the model training process, you may run `tensorboard --logdir output/`
+
+
+## Acknowledgements 🙏
+
+- Our codebase is heavily built on top of [3D Gaussians](https://github.com/graphdeco-inria/gaussian-splatting) and [4D Gaussians](https://github.com/hustvl/4DGaussians)
+- Our renderer uses [Gsplat](https://github.com/nerfstudio-project/gsplat)
+- Many robot models that we use come directyl from [Mujoco Menagerie](https://github.com/google-deepmind/mujoco_menagerie), we thank them for providing a diverse and clean repository of robot models. 
+
+
+
+## TODO 🚸
+
+- [ ] Fix the bug that tampers with training of some robots midway through training, some parameters were changed during the code clean-up, so please bear with us while we try to address this