Skip to content

Commit

Permalink
Add links to SB3 and rl_zoo3 in dataloading tutorial (#117)
Browse files Browse the repository at this point in the history
* Specify SB3 and rl-zoo3 with links

* Fix link syntax and add links to PyTorch and DataLoader
  • Loading branch information
elliottower authored Jul 18, 2023
1 parent 0508462 commit 41a22a0
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions docs/tutorials/using_datasets/behavioral_cloning.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@
=========================================
"""
# %%%
# We present here how to perform behavioral cloning on a Minari dataset using PyTorch.
# We present here how to perform behavioral cloning on a Minari dataset using `PyTorch <https://pytorch.org/>`_.
# We will start generating the dataset of the expert policy for the `CartPole-v1 <https://gymnasium.farama.org/environments/classic_control/cart_pole/>`_ environment, which is a classic control problem.
# The objective is to balance the pole on the cart, and we receive a reward of +1 for each successful timestep.

# %%
# Policy training
# ~~~~~~~~~~~~~~~~~~~
# To train the expert policy, we use the library rl_zoo3.
# To train the expert policy, we use `SB3 <https://github.com/DLR-RM/stable-baselines3>`_'s <rl-zoo3 <https://github.com/DLR-RM/rl-baselines3-zoo>`_ library.
# After installing the library with ``pip install rl_zoo3``,
# we train a PPO agent on the environment with the following command:
#
Expand Down Expand Up @@ -97,7 +97,7 @@ def forward(self, x):

# %%
# In this scenario, the output dimension will be two, as previously mentioned. As for the input dimension, it will be four, corresponding to the observation space of ``CartPole-v1``.
# Our next step is to load the dataset and set up the training loop. The ``MinariDataset`` is compatible with the PyTorch Dataset API, allowing us to load it directly using PyTorch DataLoader.
# Our next step is to load the dataset and set up the training loop. The ``MinariDataset`` is compatible with the PyTorch Dataset API, allowing us to load it directly using `PyTorch DataLoader <https://pytorch.org/docs/stable/data.html>`_.
# However, since each episode can have a varying length, we need to pad them.
# To achieve this, we can utilize the `collate_fn <https://pytorch.org/docs/stable/data.html#working-with-collate-fn>`_ feature of PyTorch DataLoader. Let's create the ``collate_fn`` function:

Expand Down

0 comments on commit 41a22a0

Please sign in to comment.