Skip to content

Commit

Permalink
[QNN EP] workaround for QNN validation bug for Tanh with uint16 quant…
Browse files Browse the repository at this point in the history
…ized output (#23432)

### Description
- Skip QNN validation for Tanh with uint16 quantized output (workaround
for QNN validation bug).
- Re-enables unit test for Tanh with uint16 quantized output.

The [QNN
documentation](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/HtpOpDefSupplement.html#tanh)
states that the output scale and offset for `ufixed_point_16` should be
(1/32768) and -32768, respectively. However, the QNN validator
incorrectly rejects these values. So, we skip validation for this
configuration of Tanh. Building an actual QNN graph with the correct
scale/offset still works.


### Motivation and Context
This QNN validation bug appeared in QNN SDK 2.28.0 and is still present
in QNN SDK 2.30.0. A previous PR disabled the corresponding unit test:
https://github.com/microsoft/onnxruntime/pull/22724/files#diff-57f590c6c548b073ba8cd8af6cf198799906f7059ea46b31cd33972ea9b01983R232
  • Loading branch information
adrianlizarraga authored Jan 21, 2025
1 parent 83cb1e4 commit 3e4c5e6
Show file tree
Hide file tree
Showing 3 changed files with 22 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,22 @@ Status SimpleOpBuilder::ProcessAttributesAndOutputs(QnnModelWrapper& qnn_model_w
if (node_unit.Domain() != kMSInternalNHWCDomain && (op_type == "DepthToSpace" || op_type == "SpaceToDepth" || op_type == "GridSample")) {
return Status::OK();
}

#if QNN_API_VERSION_MAJOR >= 2 && QNN_API_VERSION_MINOR >= 21 && QNN_API_VERSION_MINOR <= 23
// Skip QNN validation for Tanh with uint16 (quantized) output.
// This gets around a Tanh QNN validation bug in QNN SDK 2.28.0 - 2.30.0.
// The QNN documentation states that the output scale and offset for ufixed_point_16 should be
// (1/32768) and -32768, respectively. However, the QNN validator incorrectly rejects these values.
if (op_type == "Tanh") {
TensorInfo output_info = {};
ORT_RETURN_IF_ERROR(qnn_model_wrapper.GetTensorInfo(node_unit.Outputs()[0], output_info));
if (output_info.qnn_data_type == QNN_DATATYPE_UFIXED_POINT_16) {
LOGS(logger, INFO) << "Skipping QNN validation for Tanh node '"
<< node_unit.Name() << "' with quantized unit16 output.";
return Status::OK();
}
}
#endif
}

std::vector<std::string> param_tensor_names;
Expand Down
2 changes: 1 addition & 1 deletion onnxruntime/core/providers/qnn/qnn_allocator.cc
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ size_t DivRoundUp(size_t a, size_t b) { // TODO is there already a helper funct
}

bool IsAligned(const void* address, size_t alignment) {
assert((alignment & alignment - 1) == 0); // alignment must be a power of two
assert((alignment & (alignment - 1)) == 0); // alignment must be a power of two
return (reinterpret_cast<uintptr_t>(address) & (alignment - 1)) == 0;
}

Expand Down
8 changes: 5 additions & 3 deletions onnxruntime/test/providers/qnn/simple_op_htp_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -230,17 +230,19 @@ TEST_F(QnnHTPBackendTests, UnaryOp_Tanh) {
}

// disabled for QNN 2.28.0.241029 backendValidateOpConfig failed
// still fails on QNN 2.28.2.
// still fails on QNN 2.28.2 and QNN 2.30.0
// QnnDsp <E> [4294967295] has incorrect Value -32768, expected equal to 0.
// QnnDsp <V> validateNativeOps node_token_6:qti.aisw:Tanh htp op validator failed 3110
// QnnDsp <V> registered validator failed => 3110
// QnnDsp <E> QnnBackend_validateOpConfig failed 3110
// QnnDsp <V> Wake up free backend (id: 1)'s thread(s)
// QnnDsp <E> Failed to validate op node_token_6 with error 0xc26
// Tests accuracy of 16-bit QDQ Tanh.
TEST_F(QnnHTPBackendTests, DISABLED_UnaryOp_Tanh_U16) {
//
// We now skip QNN validation as a workaround for QNN SDK 2.28.0 to 2.30.0
TEST_F(QnnHTPBackendTests, UnaryOp_Tanh_U16) {
RunQDQOpTest<uint16_t>("Tanh",
{TestInputDef<float>({1, 2, 3}, false, GetFloatDataInRange(-10.0f, 10.0f, 6))},
{TestInputDef<float>({1, 2, 64}, false, GetFloatDataInRange(-10.0f, 10.0f, 128))},
{},
13,
ExpectedEPNodeAssignment::All,
Expand Down

0 comments on commit 3e4c5e6

Please sign in to comment.