Multiplication following a int8 transposed convolutions isn't constant folded

[lgeiger]

When converting a model that used int8 quantization aware training, conversion of transposed convolutions followed by a scalar multiplication fails.

The converter isn't able to correctly constant fold the per-tensor fake quantized weights and the scalar multiplication which is a common patter when using transposed convolutions followed by batch normalisation layers. This is a follow-up issue to #53766

1. System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS / Ubuntu
• TensorFlow installation (pip package or built from source): pip
• TensorFlow library (version, if pip package or github SHA, if built from source): 2.10.0 / 2.11.0-dev20220913

2. Code

A minimal reproduction of the issue is available in this notebook. Re-run the notebook to show netron visualisations showing the conversion problem.

[mohantym]

Hi @lgeiger !
Thanks for bring it up.
It is replicating in nightly version as you pointed but not in 2.10 stable release.
Could we stick with 2.10 (stable version) for a while.
Thank you!

[lgeiger]

@mohantym Thanks for the fast reply. However, I am able to reproduce this issue in 2.10, in fact this is the version which I am also using locally. Checkout this notebook.

With per-tensor fake quantization I am getting the following TFLite graph:

However the multiplication should be fused into the transposed convolution like it is done when not using any weight quantization:

I get the same result also when re-running your linked notebook.

[mohantym]

Hi @lgeiger !
Can you check again with tf-nightly. I deleted the run time and re-run the code in nightly ( issue seems to be fixed in nightly though , I think it might be specific to mac os )

@sachinprasadhs ! Could you look at this issue. Attached 2.9, 2.10 and nightly for reference.

Thank you!

[lgeiger]

Can you check again with tf-nightly. I deleted the run time and re-run the code in nightly ( issue seems to be fixed in nightly though , I think it might be specific to mac os )

I double checked by re-running your colab notebook and the issue persists even on nightly 2.11.0-dev20220915. The scalar multiplication of a transposed convolutions with per-tensor quantized weights is still not fused correctly which results in dequantization and multiplications operation in the graph so that the transposed convolution is executed in float instead of in int8.

[pjpratik]

Issue exists in TF 2.11 and TF Nightly 2.12.0-dev20230111 as well. Please find the gist here. Thank you.

[pkgoogle]

Hi,

Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.

The TFLite team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.

Please follow the release notes to stay up to date with the latest developments which are happening in the TFLite space.

[lgeiger]

@pkgoogle Thanks for the response. I double checked and the issue still persists in the latest TF nightly. See this notebook.

[pkgoogle]

Hi @lgeiger,

Thanks for raising this issue. Are you aware of AI-Edge-Torch? As we believe this issue is better supported by and more relevant to AI-Edge-Torch we are moving your issue there. Please follow progress here.

Let us know if you have any questions. Thanks.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No