Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OCP FP8 formats #3399

Merged
merged 27 commits into from
Sep 13, 2024
Merged

Add OCP FP8 formats #3399

merged 27 commits into from
Sep 13, 2024

Conversation

CharlieL7
Copy link
Collaborator

@CharlieL7 CharlieL7 commented Aug 23, 2024

@CharlieL7 CharlieL7 self-assigned this Aug 23, 2024
@CharlieL7 CharlieL7 requested a review from causten as a code owner August 23, 2024 20:15
@CharlieL7 CharlieL7 marked this pull request as draft August 23, 2024 20:15
Copy link

codecov bot commented Aug 27, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 11 lines in your changes missing coverage. Please review.

Project coverage is 92.04%. Comparing base (8ff42e8) to head (5451f91).
Report is 164 commits behind head on develop.

Files with missing lines Patch % Lines
src/quantization.cpp 18.18% 9 Missing ⚠️
src/include/migraphx/op/quant_convolution.hpp 50.00% 1 Missing ⚠️
src/include/migraphx/op/quant_dot.hpp 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3399      +/-   ##
===========================================
- Coverage    92.07%   92.04%   -0.04%     
===========================================
  Files          504      505       +1     
  Lines        20689    20699      +10     
===========================================
+ Hits         19050    19052       +2     
- Misses        1639     1647       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* these were previously disabled for fp8E4M3FNUZ, need to also disable
  for the new formats
@CharlieL7 CharlieL7 marked this pull request as ready for review September 5, 2024 15:29
@CharlieL7 CharlieL7 added the roadmap Tasks to finish for a release label Sep 5, 2024
@TedThemistokleous TedThemistokleous added the FP8 issues related to FP8 implemenation label Sep 5, 2024
Copy link
Collaborator

@TedThemistokleous TedThemistokleous left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make the TODOs separate issues so we can track them.

I kicked off CI for the other build checks. Just ensure CI is working and you add tickets and I'll approve.

@TedThemistokleous
Copy link
Collaborator

CI failure seems unrelated to your changes. Getting a lot of "Failed to fetch" errors.

@TedThemistokleous
Copy link
Collaborator

Approved.. I'll rekick CI but that MLIR issue seems like its server related and not build.

@TedThemistokleous TedThemistokleous linked an issue Sep 6, 2024 that may be closed by this pull request
8 tasks
@migraphx-bot
Copy link
Collaborator

Test Batch Rate new
5451f9
Rate old
b87355
Diff Compare
torchvision-resnet50 64 3,249.70 3,251.19 -0.05%
torchvision-resnet50_fp16 64 6,989.55 6,988.29 0.02%
torchvision-densenet121 32 2,432.87 2,433.72 -0.03%
torchvision-densenet121_fp16 32 4,105.26 4,109.85 -0.11%
torchvision-inceptionv3 32 1,635.92 1,636.15 -0.01%
torchvision-inceptionv3_fp16 32 2,739.66 2,740.01 -0.01%
cadene-inceptionv4 16 776.75 776.10 0.08%
cadene-resnext64x4 16 807.38 808.31 -0.12%
slim-mobilenet 64 7,453.43 7,462.97 -0.13%
slim-nasnetalarge 64 208.28 208.06 0.10%
slim-resnet50v2 64 3,435.76 3,344.04 2.74%
bert-mrpc-onnx 8 1,154.16 1,149.25 0.43%
bert-mrpc-tf 1 309.90 307.92 0.64%
pytorch-examples-wlang-gru 1 430.90 414.16 4.04% 🔆
pytorch-examples-wlang-lstm 1 383.72 376.68 1.87%
torchvision-resnet50_1 1 801.78 814.48 -1.56%
cadene-dpn92_1 1 395.99 400.81 -1.20%
cadene-resnext101_1 1 383.40 380.42 0.78%
onnx-taau-downsample 1 344.90 344.67 0.07%
dlrm-criteoterabyte 1 35.05 35.06 -0.04%
dlrm-criteoterabyte_fp16 1 58.12 58.10 0.02%
agentmodel 1 8,081.40 8,179.29 -1.20%
unet_fp16 2 58.03 57.98 0.08%
resnet50v1_fp16 1 932.44 985.87 -5.42% 🔴
resnet50v1_int8 1 949.16 935.52 1.46%
bert_base_cased_fp16 64 1,154.07 1,154.71 -0.05%
bert_large_uncased_fp16 32 355.86 355.97 -0.03%
bert_large_fp16 1 211.27 211.90 -0.30%
distilgpt2_fp16 16 2,162.77 2,159.23 0.16%
yolov5s 1 530.35 540.08 -1.80%
tinyllama 1 43.40 43.46 -0.13%
vicuna-fastchat 1 170.26 179.52 -5.16% 🔴
whisper-tiny-encoder 1 417.83 418.25 -0.10%
whisper-tiny-decoder 1 433.46 433.10 0.08%

This build is not recommended to merge 🔴

@migraphx-bot
Copy link
Collaborator


     ✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

     ✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     ✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-resnext101_1: PASSED: MIGraphX meets tolerance

     ✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     ✅ agentmodel: PASSED: MIGraphX meets tolerance

     ✅ unet: PASSED: MIGraphX meets tolerance

     ✅ resnet50v1: PASSED: MIGraphX meets tolerance

     ✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output


     ✅ bert_large: PASSED: MIGraphX meets tolerance

     ✅ yolov5s: PASSED: MIGraphX meets tolerance

     ✅ tinyllama: PASSED: MIGraphX meets tolerance

     ✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     ✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance

@causten causten merged commit 9dcea5c into develop Sep 13, 2024
46 of 48 checks passed
@causten causten deleted the fp8_ocp branch September 13, 2024 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FP8 issues related to FP8 implemenation roadmap Tasks to finish for a release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update MIGraphX FP8 to support OCP types
5 participants