[ GPU ] Update android_test.sh to enable unittests for GPU @open sesame 11/04 09:08 #2773

EunjuYang · 2024-10-25T03:30:22Z

This commit updates android_test.sh to enable unittests for GPU
- create nntrainer_opencl_kernels folder
- add some command lines to push nngolden data to android
unittest_layers.tar.gz is updated to add nnlayer golden data for GPU layers.
This commit updates how-to-use-testcases.md file, including guide for running test cases for OpenCL on Android.

Note

This PR doesn't resolve all unit-test but aims to build a common environment to run GPU-related unittests.
There are still 12 unittest failures. Please refer to [ Unittest / GPU ] GPU Unittest failure (12 tests ) in unittest_layers #2774

Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

taos-ci · 2024-10-25T03:30:25Z

📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2773. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

taos-ci · 2024-10-25T03:30:30Z

cibot: @EunjuYang, The last line of a text file must have a newline character. Please append a new line at the end of the line in docs/how-to-use-testcases.md.

taos-ci

@EunjuYang, 💯 All CI checkers are successfully verified. Thanks.

DonghakPark · 2024-10-25T04:48:34Z

test/jni/Android.mk

@@ -458,7 +458,6 @@ LOCAL_SRC_FILES := \
 	 ../unittest/layers/unittest_layers_flatten.cpp \
 	 ../unittest/layers/unittest_layers_activation.cpp \
 	 ../unittest/layers/unittest_layers_addition.cpp \
-	 ../unittest/layers/unittest_layers_addition_cl.cpp \


Question) is there any reason to remove this from unittest_layers build ?

This ../unittest/layers/unittest_layers_addition_cl.cpp should be added after #2752.
If not, NDK build returns error, for the addition layer for OpenCL is not implemented in the current upstream. I wanted to make upstream run as it is.
By the way... it would be better to change this PR [Wait for #2752], while removing this change.

i see, thank you for detail explain

By the way... it would be better to change this PR [Wait for https://github.com//pull/2752], while removing this change.

Yes i would be good!! or i think you can simply rebase #2752 or cherry pick some commit's ( if there are some dependency)

rebase sample command
git pull --rebase upstream refs/pull/2752/head:

Sounds good! I will rebase #2752. Thank you ☺

@DonghakPark
I tested rebasing on #2752, but the PR itself is not well rebased on the main; It makes this PR jammed.
Instead of rebasing on it, I updated the Android.mk to leave line you mentioned as commented out. Could you please check the update?
If #2752 is merged or it resolves the rebasing issue, I will update this PR as well.

If so, it seems to be good. It may reduce inconvenience

taos-ci

@EunjuYang, 💯 All CI checkers are successfully verified. Thanks.

taos-ci

@EunjuYang, 💯 All CI checkers are successfully verified. Thanks.

djeong20 · 2024-10-28T01:41:50Z

tools/android_test.sh

-# $ adb push res/ /data/local/tmp/nntr_android_test
+# meson build will unzip golden data for the unit tests
+cd ../../../
+meson build


it would be nice to add some basic flags to get the proper golden data (e.g., -Denable-fp16=true, -Denable-neon=true, -Denable-opencl=true)

Good point!! for more information we use below flags for build android_package

-Dplatform=android -Dopenblas-num-threads=1 -Denable-tflite-interpreter=false -Denable-tflite-backbone=false -Denable-fp16=true -Denable-neon=true -Domp-num-threads=1 -Denable-opencl=true -Dhgemm-experimental-kernel=false

Thank you for the comments, all! It would be better to apply the same options for the android build. I will apply it 😀

@DonghakPark I applied every option you suggested except for the android option, formeson.build doesn't create resource for unittest with -Dplatofrm=android option

Please refer to

nntrainer/meson.build

Line 439 in 63f9875

warning('test is not supported in android build, test skipped')

Thank you for considering my opinion.
I think we need to ensure that testing is enabled in the future, even for cases of Android as well. (As there are increasingly more examples utilizing Android.)
I will create an issue for this matter.

Enable Android Test with meson build #2775

baek2sm · 2024-10-28T04:23:11Z

Don't we need a script to make golden data as well?

EunjuYang · 2024-10-29T00:25:53Z

Don't we need a script to make golden data as well?

Thank you for your comment. The script for the golden data was already updated in gen_lsyer_tests.py in previous PRs. However, they were tested by creating the golden data manually and pushing them to test environment.
To make it easier and a common test tool, this PR updates unittest_layers.tar.gz and android_test.sh. If I missed some steps, please let me know.

FYI, I attach some links for the golden layer generation parts additionally added in this update :

nntrainer/test/input_gen/gen_layer_tests.py

Line 927 in 63f9875

class RMSNorm(tf.keras.layers.Layer):

nntrainer/test/input_gen/gen_layer_tests.py

Line 917 in 63f9875

    
           def reshape_tensor(tensor, batch_size, input_channel, input_height, input_width):

nntrainer/test/input_gen/gen_layer_tests.py

Line 885 in 63f9875

def swiglu(inputs):

taos-ci

@EunjuYang, 💯 All CI checkers are successfully verified. Thanks.

baek2sm

LGTM

djeong20 · 2024-10-31T01:57:55Z

tools/android_test.sh

-# $ adb push res/ /data/local/tmp/nntr_android_test
+# meson build will unzip golden data for the unit tests
+cd ../../../
+meson build -Dopenblas-num-threads=1  \


there might be cases where the build directory already exists. we should handle that case!

Great point! I will update the script. Thanks!

- This commit updates android_test.sh to enable unittests for GPU - This commit updates `how-to-use-testcases.md` file, including guide for running test cases for OpenCL on Android. - This commit remove add kernel from Android.mk in order to make it enabled (commented out) - unittest_layers.tar.gz is updated to add nnlayer golden data for GPU layers. **Self evaluation**: 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Eunju Yang <[email protected]>

taos-ci

@EunjuYang, 💯 All CI checkers are successfully verified. Thanks.

- This commit is related to nnstreamer#2774. - The test `setPropertiesInvalid_n_gpu` was failed with LayerPropertySemantics, which tests negative case for some layers not implemented for GPU. - This test should be enabled later. Self evaluation: Build test: [X]Passed [ ]Failed [ ]Skipped Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Eunju Yang <[email protected]>

taos-ci

@EunjuYang, 💯 All CI checkers are successfully verified. Thanks.

djeong20

LGTM!

skykongkong8

This resolves many inconveniences that I have been working on. Thanks!

EunjuYang marked this pull request as ready for review October 25, 2024 03:30

EunjuYang requested review from myungjoo, jijoongmoon, again4you, jaeyun-jung, leemgs, wooksong, gichan-jang, anyj0527, lhs8928, songgot, jihochu, DonghakPark, SeoHyungjun, baek2sm, skykongkong8 and djeong20 as code owners October 25, 2024 03:30

github-actions bot added the Need Review label Oct 25, 2024

taos-ci approved these changes Oct 25, 2024

View reviewed changes

EunjuYang force-pushed the gpu_unittest_setting branch from 48ca2f9 to f8d520f Compare October 25, 2024 04:17

DonghakPark reviewed Oct 25, 2024

View reviewed changes

taos-ci approved these changes Oct 25, 2024

View reviewed changes

EunjuYang mentioned this pull request Oct 25, 2024

[ Unittest / GPU ] GPU Unittest failure (12 tests ) in unittest_layers #2774

Closed

EunjuYang force-pushed the gpu_unittest_setting branch from f8d520f to 7bcc1f4 Compare October 25, 2024 06:54

EunjuYang changed the title ~~[ GPU ] Update android_test.sh to enable unittests for GPU~~ [Wait for #2752] [ GPU ] Update android_test.sh to enable unittests for GPU Oct 25, 2024

EunjuYang force-pushed the gpu_unittest_setting branch from 7bcc1f4 to e00d205 Compare October 25, 2024 07:09

EunjuYang changed the title ~~[Wait for #2752] [ GPU ] Update android_test.sh to enable unittests for GPU~~ [ GPU ] Update android_test.sh to enable unittests for GPU Oct 25, 2024

taos-ci approved these changes Oct 25, 2024

View reviewed changes

djeong20 approved these changes Oct 28, 2024

View reviewed changes

EunjuYang force-pushed the gpu_unittest_setting branch from e00d205 to 84c9ca9 Compare October 29, 2024 01:08

taos-ci approved these changes Oct 29, 2024

View reviewed changes

baek2sm approved these changes Oct 31, 2024

View reviewed changes

github-actions bot added PR/READY2MERGE and removed Need Review labels Oct 31, 2024

djeong20 reviewed Oct 31, 2024

View reviewed changes

EunjuYang force-pushed the gpu_unittest_setting branch from 84c9ca9 to 71f3a4a Compare November 1, 2024 03:26

taos-ci approved these changes Nov 1, 2024

View reviewed changes

jijoongmoon changed the title ~~[ GPU ] Update android_test.sh to enable unittests for GPU~~ [ GPU ] Update android_test.sh to enable unittests for GPU @open sesame 11/04 09:08 Nov 4, 2024

EunjuYang force-pushed the gpu_unittest_setting branch from 71f3a4a to fd95c4f Compare November 4, 2024 00:18

taos-ci approved these changes Nov 4, 2024

View reviewed changes

djeong20 approved these changes Nov 4, 2024

View reviewed changes

skykongkong8 approved these changes Nov 6, 2024

View reviewed changes

DonghakPark approved these changes Nov 6, 2024

View reviewed changes

jijoongmoon merged commit 158d18d into nnstreamer:main Nov 15, 2024
40 checks passed

EunjuYang mentioned this pull request Nov 19, 2024

[Wait for 2759][GPU/OpenCL] Initial version of LM Head layer with OpenCl ops and updated Addition Layer with latest pipeline changes. @open sesame 12/02 16:21 #2752

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ GPU ] Update android_test.sh to enable unittests for GPU @open sesame 11/04 09:08 #2773

[ GPU ] Update android_test.sh to enable unittests for GPU @open sesame 11/04 09:08 #2773

EunjuYang commented Oct 25, 2024 •

edited

Loading

taos-ci commented Oct 25, 2024

taos-ci commented Oct 25, 2024

taos-ci left a comment

DonghakPark Oct 25, 2024

EunjuYang Oct 25, 2024 •

edited

Loading

DonghakPark Oct 25, 2024 •

edited

Loading

EunjuYang Oct 25, 2024 •

edited

Loading

EunjuYang Oct 25, 2024

DonghakPark Oct 29, 2024

taos-ci left a comment

taos-ci left a comment

djeong20 Oct 28, 2024

DonghakPark Oct 29, 2024

EunjuYang Oct 29, 2024

EunjuYang Oct 29, 2024 •

edited

Loading

DonghakPark Oct 29, 2024

baek2sm commented Oct 28, 2024

EunjuYang commented Oct 29, 2024

taos-ci left a comment

baek2sm left a comment

djeong20 Oct 31, 2024

EunjuYang Nov 1, 2024

taos-ci left a comment

taos-ci left a comment

djeong20 left a comment

skykongkong8 left a comment

[ GPU ] Update android_test.sh to enable unittests for GPU @open sesame 11/04 09:08 #2773

[ GPU ] Update android_test.sh to enable unittests for GPU @open sesame 11/04 09:08 #2773

Conversation

EunjuYang commented Oct 25, 2024 • edited Loading

taos-ci commented Oct 25, 2024

taos-ci commented Oct 25, 2024

taos-ci left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EunjuYang Oct 25, 2024 • edited Loading

Choose a reason for hiding this comment

DonghakPark Oct 25, 2024 • edited Loading

Choose a reason for hiding this comment

EunjuYang Oct 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

taos-ci left a comment

Choose a reason for hiding this comment

taos-ci left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EunjuYang Oct 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

baek2sm commented Oct 28, 2024

EunjuYang commented Oct 29, 2024

taos-ci left a comment

Choose a reason for hiding this comment

baek2sm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

taos-ci left a comment

Choose a reason for hiding this comment

taos-ci left a comment

Choose a reason for hiding this comment

djeong20 left a comment

Choose a reason for hiding this comment

skykongkong8 left a comment

Choose a reason for hiding this comment

EunjuYang commented Oct 25, 2024 •

edited

Loading

EunjuYang Oct 25, 2024 •

edited

Loading

DonghakPark Oct 25, 2024 •

edited

Loading

EunjuYang Oct 25, 2024 •

edited

Loading

EunjuYang Oct 29, 2024 •

edited

Loading