Skip to content

Commit

Permalink
amend
Browse files Browse the repository at this point in the history
  • Loading branch information
vmoens committed Apr 29, 2024
1 parent fce8ec7 commit 6f9c672
Showing 1 changed file with 15 additions and 0 deletions.
15 changes: 15 additions & 0 deletions docs/source/reference/data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,8 @@ using the following components:
RoundRobinWriter
TensorDictRoundRobinWriter
TensorDictMaxValueWriter
TED2Flat
Flat2TED

Storage choice is very influential on replay buffer sampling latency, especially
in distributed reinforcement learning settings with larger data volumes.
Expand Down Expand Up @@ -518,6 +520,19 @@ should have a considerably lower memory footprint than observations, for instanc
This format eliminates any ambiguity regarding the matching of an observation with
its action, info, or done state.

Flattening TED to reduce memory consumption
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TED copies the observations twice in memory, which can impact the feasibility of using this format
in practice. Since it is being used mostly for ease of representation, one can store the data
in a flat manner but represent it as TED during training.

This is particularly useful when serializing replay buffers:
For instance, the :class:`~torchrl.data.TED2Flat` class ensures that a TED-formatted data
structure is flattened before being written to disk, whereas the :class:`~torchrl.data.Flat2TED`
load hook will unflatten this structure during deserialization.


Dimensionality of the Tensordict
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down

0 comments on commit 6f9c672

Please sign in to comment.