Skip to content

Commit

Permalink
Update ddp_tutorial.rst (#2516)
Browse files Browse the repository at this point in the history
1. add `dist.destroy_process_group()` in example code block
2. modify the link syntax error about torchrun
  • Loading branch information
HUGHNew authored Aug 3, 2023
1 parent bf728cb commit b4e6207
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion intermediate_source/ddp_tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -340,11 +340,12 @@ Let's still use the Toymodel example and create a file named ``elastic_ddp.py``.
labels = torch.randn(20, 5).to(device_id)
loss_fn(outputs, labels).backward()
optimizer.step()
dist.destroy_process_group()
if __name__ == "__main__":
demo_basic()
One can then run a `torch elastic/torchrun<https://pytorch.org/docs/stable/elastic/quickstart.html>`__ command
One can then run a `torch elastic/torchrun <https://pytorch.org/docs/stable/elastic/quickstart.html>`__ command
on all nodes to initialize the DDP job created above:

.. code:: bash
Expand Down

0 comments on commit b4e6207

Please sign in to comment.