Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ONNX export] Add depth-estimation w/ DPT+GLPN #1529

Merged
merged 3 commits into from
Nov 14, 2023
Merged

Conversation

xenova
Copy link
Contributor

@xenova xenova commented Nov 11, 2023

What does this PR do?

This PR adds the ability to export DPT+GLPN models to ONNX for depth-estimation, unblocking huggingface/transformers.js#350.

... and it crosses DPT+GLPN off this list.

Running

optimum-cli export onnx -m hf-internal-testing/tiny-random-DPTModel o --task depth-estimation

now shows:

...
Validating ONNX model o/model.onnx...
        -[✓] ONNX model output names match reference model (predicted_depth)
        - Validating ONNX Model output "predicted_depth":
                -[✓] (2, 32, 32) matches (2, 32, 32)
                -[✓] all values close (atol: 1e-05)
The ONNX export succeeded and the exported model was saved at: o

Similarly,

optimum-cli export onnx -m hf-internal-testing/tiny-random-GLPNForDepthEstimation o

produces:

...
Validating ONNX model o/model.onnx...
        -[✓] ONNX model output names match reference model (predicted_depth)
        - Validating ONNX Model output "predicted_depth":
                -[✓] (2, 64, 64) matches (2, 64, 64)
                -[✓] all values close (atol: 1e-05)
The ONNX export succeeded and the exported model was saved at: o

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@xenova xenova requested a review from fxmarty November 11, 2023 01:26
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Nov 11, 2023

The documentation is not available anymore as the PR was closed or merged.

@xenova xenova changed the title [ONNX export] Add depth-estimation w/ dpt [ONNX export] Add depth-estimation w/ DPT+GLPN Nov 11, 2023
@xenova
Copy link
Contributor Author

xenova commented Nov 13, 2023

Note: The DPT conversion fails when a custom backbone (e.g., dinov2) is used. The associated PR was merged ~3 hours ago. This will need a closer look.

Copy link
Contributor

@fxmarty fxmarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thank you! About the custom backbone, does this need to be fixed in this PR (aka, are large models using it)?

@xenova
Copy link
Contributor Author

xenova commented Nov 14, 2023

are large models using it

Nope, all the latest dpt-dinov2 models, highlighted in red below, have very few downloads (the most is 155, and the rest average around 10). The main two checkpoints from Intel, highlighted in green, have significantly more downloads.

image

So, I think it's okay to merge for now, then add support for these other models at a later date. I have no doubt that they will grow in popularity, considering they are released under the facebook org.

@fxmarty
Copy link
Contributor

fxmarty commented Nov 14, 2023

Sounds good!

@fxmarty fxmarty merged commit e3b7efb into main Nov 14, 2023
59 of 70 checks passed
@fxmarty fxmarty deleted the add-depth-estimation branch November 14, 2023 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants