Skip to content

Commit

Permalink
[onert_micro] workaround for tanh for tizenrt (#14094)
Browse files Browse the repository at this point in the history
* [onert_micro] workaround for tanh for tizenrt

- TizenRT platform produces tanh(x) NaN for large x
- reason behind this patch is that tanh(x) is almost 1 for x > 15

ONE-DCO-1.0-Signed-off-by: Chunseok Lee <[email protected]>

* fix format
  • Loading branch information
chunseoklee authored Sep 27, 2024
1 parent a163a66 commit 0aa5ce0
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 1 deletion.
13 changes: 12 additions & 1 deletion onert-micro/onert-micro/include/pal/common/PALGRUCommon.h
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,18 @@ void calculateGRU(const float *input_data, const float *weight_input_data,
}
for (int i = 0; i < num_elements; ++i)
{
second_hidden_part[i] = std::tanh(second_hidden_part[i]);
if (second_hidden_part[i] > 19)
{
second_hidden_part[i] = 1;
}
else if (second_hidden_part[i] < -19)
{
second_hidden_part[i] = -1;
}
else
{
second_hidden_part[i] = std::tanh(second_hidden_part[i]);
}
}

// If train mode - save mul input (left and right)
Expand Down
9 changes: 9 additions & 0 deletions onert-micro/onert-micro/include/pal/common/PALGRUWeightGrad.h
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,15 @@ void calculateGRUWeightGrads(
float tanh_grad_value;
{
float tanh = std::tanh(tanh_data[i]);
if (tanh_data[i] > 19)
{
tanh = 1;
}
else if (tanh_data[i] < -19)
{
tanh = -1;
}

tanh_grad_value = (1 - tanh * tanh) * right_middle_mul;
}

Expand Down

0 comments on commit 0aa5ce0

Please sign in to comment.