Synergistic Reinforcement and Imitation Learning (SRIL) for Vision-driven Autonomous Flight of UAV Along River

Author: Zihan Wang, Jianwen Li and Nina Mahmoudian

Mechanical Engineering, Purdue University

IROS2024-480p.mp4

Installation

• The Riverine Simulation Environment has only been tested on Ubuntu 20.04 and Ubuntu 22.04.

• The Unity riverine environments can be downloaded from Google Drive link, then unzipped to the folder 'riverine_simulation'.

• The VAE model can be downloaded from this link, then put it to 'encoder/models/vae-sim-rgb-all.pth'.

• Miniconda is encouraged to build the virtual python environment, and Python 3.9 is the supported version.

conda create -n SRIL python==3.9 && conda activate SRIL

• Install package dependencies

pip install -r requirements.txt

• If have error installing gym during the above process, install the specific version of setuptools then pip install again

pip install -U setuptools==65.5.0 

• Run the riverine_simulation/collect_demonstrations.py to control the UAV agent in the Unity riverine environment to test successful installation. Keys W and S control camera altitude, A and D control yaw, I and K control longitudinal movement, J and L control latitudinal movement.

UnityRiverineEnvironment-demo-cropped-compressed.mp4

• If have protobuf error running the Unity environment, downgrade the protobuf version

pip install -U protobuf==3.20

• If you want to play the CliffCircular environment, run python -m cliff_circular.cliffcircular or python -m cliff_circular.cliffcircular_gym.

CliffCircular-human-control-cropped.mp4

Getting started

• To train BC expert in the CliffCircular environment, run

python -m sril.train_bc_cliff

• To train BC expert in the Unity riverine environment, run

python -m sril.train_bc_riverine

• To train PPO+DynamicBC agent in the CliffCircular environment, run

python -m sril.cliff_gym_trainer

• To train PPO+DynamicBC agent in the Unity riverine environment, run

python -m sril.unity_gym_trainer

• To check the training log, run tensorboard --logdir sril/ppo_river_tensorboard/ or tensorboard --logdir sril/ppo_cliff_tensorboard/