Skip to content

Commit

Permalink
updates
Browse files Browse the repository at this point in the history
  • Loading branch information
@bottler
bottler committed May 9, 2023
1 parent f0e5244 commit bae55d0
Show file tree
Hide file tree
Showing 82 changed files with 292 additions and 436 deletions.
2 changes: 1 addition & 1 deletion docs/batching.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,4 @@ <h2><a class="anchor" aria-hidden="true" id="batch-modes-for-meshes"></a><a href
<h2><a class="anchor" aria-hidden="true" id="use-cases-for-batch-modes"></a><a href="#use-cases-for-batch-modes" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Use cases for batch modes</h2>
<p>The need for different mesh batch modes is inherent to the way PyTorch operators are implemented. To fully utilize the optimized PyTorch ops, the <a href="https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/structures/meshes.py">Meshes</a> data structure allows for efficient conversion between the different batch modes. This is crucial when aiming for a fast and efficient training cycle. An example of this is <a href="https://github.com/facebookresearch/meshrcnn">Mesh R-CNN</a>. Here, in the same forward pass different parts of the network assume different inputs, which are computed by converting between the different batch modes. In particular, <a href="https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/ops/vert_align.py">vert_align</a> assumes a <em>padded</em> input tensor while immediately after <a href="https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/ops/graph_conv.py">graph_conv</a> assumes a <em>packed</em> input tensor.</p>
<p><img src="assets/meshrcnn.png" alt="meshrcnn" width="700" align="middle" /></p>
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Jeremy Reizenstein</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/datasets"><span class="arrow-prev">← </span><span>Data loaders</span></a><a class="docs-next button" href="/docs/cubify"><span>Cubify</span><span class="arrow-next"> →</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#batch-modes-for-meshes">Batch modes for meshes</a></li><li><a href="#use-cases-for-batch-modes">Use cases for batch modes</a></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Jeremy Reizenstein</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/datasets"><span class="arrow-prev">← </span><span>Data loaders</span></a><a class="docs-next button" href="/docs/cubify"><span>Cubify</span><span class="arrow-next"> →</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#batch-modes-for-meshes">Batch modes for meshes</a></li><li><a href="#use-cases-for-batch-modes">Use cases for batch modes</a></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2023 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
2 changes: 1 addition & 1 deletion docs/batching/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,4 @@ <h2><a class="anchor" aria-hidden="true" id="batch-modes-for-meshes"></a><a href
<h2><a class="anchor" aria-hidden="true" id="use-cases-for-batch-modes"></a><a href="#use-cases-for-batch-modes" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Use cases for batch modes</h2>
<p>The need for different mesh batch modes is inherent to the way PyTorch operators are implemented. To fully utilize the optimized PyTorch ops, the <a href="https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/structures/meshes.py">Meshes</a> data structure allows for efficient conversion between the different batch modes. This is crucial when aiming for a fast and efficient training cycle. An example of this is <a href="https://github.com/facebookresearch/meshrcnn">Mesh R-CNN</a>. Here, in the same forward pass different parts of the network assume different inputs, which are computed by converting between the different batch modes. In particular, <a href="https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/ops/vert_align.py">vert_align</a> assumes a <em>padded</em> input tensor while immediately after <a href="https://github.com/facebookresearch/pytorch3d/blob/main/pytorch3d/ops/graph_conv.py">graph_conv</a> assumes a <em>packed</em> input tensor.</p>
<p><img src="assets/meshrcnn.png" alt="meshrcnn" width="700" align="middle" /></p>
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Jeremy Reizenstein</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/datasets"><span class="arrow-prev">← </span><span>Data loaders</span></a><a class="docs-next button" href="/docs/cubify"><span>Cubify</span><span class="arrow-next"> →</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#batch-modes-for-meshes">Batch modes for meshes</a></li><li><a href="#use-cases-for-batch-modes">Use cases for batch modes</a></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Jeremy Reizenstein</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/datasets"><span class="arrow-prev">← </span><span>Data loaders</span></a><a class="docs-next button" href="/docs/cubify"><span>Cubify</span><span class="arrow-next"> →</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#batch-modes-for-meshes">Batch modes for meshes</a></li><li><a href="#use-cases-for-batch-modes">Use cases for batch modes</a></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2023 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
4 changes: 2 additions & 2 deletions docs/cameras.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ <h4><a class="anchor" aria-hidden="true" id="perspectivecameras-orthographiccame
<span class="hljs-comment"># Screen space camera</span>
image_size = ((<span class="hljs-number">128</span>, <span class="hljs-number">256</span>),) <span class="hljs-comment"># (h, w)</span>
fcl_screen = (<span class="hljs-number">76.8</span>,) <span class="hljs-comment"># fcl_ndc * min(image_size) / 2</span>
prp_screen = ((<span class="hljs-number">115.2</span>, <span class="hljs-number">48</span>), ) <span class="hljs-comment"># w / 2 - px_ndc * min(image_size) / 2, h / 2 - py_ndc * min(image_size) / 2</span>
prp_screen = ((<span class="hljs-number">115.2</span>, <span class="hljs-number">32</span>), ) <span class="hljs-comment"># w / 2 - px_ndc * min(image_size) / 2, h / 2 - py_ndc * min(image_size) / 2</span>
cameras_screen = PerspectiveCameras(focal_length=fcl_screen, principal_point=prp_screen, in_ndc=<span class="hljs-literal">False</span>, image_size=image_size)
</code></pre>
<p>The relationship between screen and NDC specifications of a camera's <code>focal_length</code> and <code>principal_point</code> is given by the following equations, where <code>s = min(image_width, image_height)</code>.
Expand All @@ -136,4 +136,4 @@ <h4><a class="anchor" aria-hidden="true" id="perspectivecameras-orthographiccame
<span class="hljs-attr">px_ndc</span> = - (px_screen - image_width / <span class="hljs-number">2.0</span>) * <span class="hljs-number">2.0</span> / s
<span class="hljs-attr">py_ndc</span> = - (py_screen - image_height / <span class="hljs-number">2.0</span>) * <span class="hljs-number">2.0</span> / s
</code></pre>
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Jeremy Reizenstein</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/renderer_getting_started"><span class="arrow-prev">← </span><span>Getting Started</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#camera-coordinate-systems">Camera Coordinate Systems</a></li><li><a href="#defining-cameras-in-pytorch3d">Defining Cameras in PyTorch3D</a></li><li><a href="#interfacing-with-the-pytorch3d-renderer">Interfacing with the PyTorch3D Renderer</a><ul class="toc-headings"><li><a href="#camera-types">Camera Types</a></li></ul></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Xiao Xuan</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/renderer_getting_started"><span class="arrow-prev">← </span><span>Getting Started</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#camera-coordinate-systems">Camera Coordinate Systems</a></li><li><a href="#defining-cameras-in-pytorch3d">Defining Cameras in PyTorch3D</a></li><li><a href="#interfacing-with-the-pytorch3d-renderer">Interfacing with the PyTorch3D Renderer</a><ul class="toc-headings"><li><a href="#camera-types">Camera Types</a></li></ul></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2023 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
4 changes: 2 additions & 2 deletions docs/cameras/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ <h4><a class="anchor" aria-hidden="true" id="perspectivecameras-orthographiccame
<span class="hljs-comment"># Screen space camera</span>
image_size = ((<span class="hljs-number">128</span>, <span class="hljs-number">256</span>),) <span class="hljs-comment"># (h, w)</span>
fcl_screen = (<span class="hljs-number">76.8</span>,) <span class="hljs-comment"># fcl_ndc * min(image_size) / 2</span>
prp_screen = ((<span class="hljs-number">115.2</span>, <span class="hljs-number">48</span>), ) <span class="hljs-comment"># w / 2 - px_ndc * min(image_size) / 2, h / 2 - py_ndc * min(image_size) / 2</span>
prp_screen = ((<span class="hljs-number">115.2</span>, <span class="hljs-number">32</span>), ) <span class="hljs-comment"># w / 2 - px_ndc * min(image_size) / 2, h / 2 - py_ndc * min(image_size) / 2</span>
cameras_screen = PerspectiveCameras(focal_length=fcl_screen, principal_point=prp_screen, in_ndc=<span class="hljs-literal">False</span>, image_size=image_size)
</code></pre>
<p>The relationship between screen and NDC specifications of a camera's <code>focal_length</code> and <code>principal_point</code> is given by the following equations, where <code>s = min(image_width, image_height)</code>.
Expand All @@ -136,4 +136,4 @@ <h4><a class="anchor" aria-hidden="true" id="perspectivecameras-orthographiccame
<span class="hljs-attr">px_ndc</span> = - (px_screen - image_width / <span class="hljs-number">2.0</span>) * <span class="hljs-number">2.0</span> / s
<span class="hljs-attr">py_ndc</span> = - (py_screen - image_height / <span class="hljs-number">2.0</span>) * <span class="hljs-number">2.0</span> / s
</code></pre>
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Jeremy Reizenstein</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/renderer_getting_started"><span class="arrow-prev">← </span><span>Getting Started</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#camera-coordinate-systems">Camera Coordinate Systems</a></li><li><a href="#defining-cameras-in-pytorch3d">Defining Cameras in PyTorch3D</a></li><li><a href="#interfacing-with-the-pytorch3d-renderer">Interfacing with the PyTorch3D Renderer</a><ul class="toc-headings"><li><a href="#camera-types">Camera Types</a></li></ul></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2022 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
</span></div></article></div><div class="docLastUpdate"><em>Last updated by Xiao Xuan</em></div><div class="docs-prevnext"><a class="docs-prev button" href="/docs/renderer_getting_started"><span class="arrow-prev">← </span><span>Getting Started</span></a></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#camera-coordinate-systems">Camera Coordinate Systems</a></li><li><a href="#defining-cameras-in-pytorch3d">Defining Cameras in PyTorch3D</a></li><li><a href="#interfacing-with-the-pytorch3d-renderer">Interfacing with the PyTorch3D Renderer</a><ul class="toc-headings"><li><a href="#camera-types">Camera Types</a></li></ul></li></ul></nav></div><footer class="nav-footer" id="footer"><section class="sitemap"><div class="footerSection"><div class="social"><a class="github-button" href="https://github.com/facebookresearch/pytorch3d" data-count-href="https://github.com/facebookresearch/pytorch3d/stargazers" data-show-count="true" data-count-aria-label="# stargazers on GitHub" aria-label="Star PyTorch3D on GitHub">pytorch3d</a></div></div></section><a href="https://opensource.facebook.com/" target="_blank" rel="noreferrer noopener" class="fbOpenSource"><img src="/img/oss_logo.png" alt="Facebook Open Source" width="170" height="45"/></a><section class="copyright">Copyright © 2023 Meta Platforms, Inc<br/>Legal:<a href="https://opensource.facebook.com/legal/privacy/" target="_blank" rel="noreferrer noopener">Privacy</a><a href="https://opensource.facebook.com/legal/terms/" target="_blank" rel="noreferrer noopener">Terms</a></section></footer></div></body></html>
Loading

0 comments on commit bae55d0

Please sign in to comment.