Intermittent incorrect results with sparse BLAS spmv function

[sidarth-narayanan-csi]

Hello All,

I am trying to run a simple SPMV program and compare the results with the CPU version. I am getting incorrect results intermittently and am not sure why. My example is very similar to the provided example and I even wait after every operation instead of using events for extra caution. I have attached the code and the compile command with this post and would appreciate any inputs on this matter.

RESULT ON PVC:

snarayanan@intel-eagle:tests$ ./spmv
Running on Intel(R) Data Center GPU Max 1100
TEST PASSED!
snarayanan@intel-eagle:tests$ ./spmv
Running on Intel(R) Data Center GPU Max 1100
TEST PASSED!
snarayanan@intel-eagle:tests$ ./spmv
Running on Intel(R) Data Center GPU Max 1100
RESULT DIFF DETECTED AT BELOW INDICES:
index = 153 || gpu_result = 1.7594586782367116e+02 || cpu_result 9.9537382428789343e+01
index = 255 || gpu_result = 1.6915528200555644e+04 || cpu_result 8.5016386081639339e+03
TEST FAILED!

RESULTS ON NVIDIA A2000:

snarayanan@rockyserver1:tests$ ./spmv
Running on NVIDIA RTX A2000 12GB
RESULT DIFF DETECTED AT BELOW INDICES:
index = 128 || gpu_result = 1.7001674668581026e+00 || cpu_result 5.8254193516886446e+00
index = 129 || gpu_result = 0.0000000000000000e+00 || cpu_result 6.2116611788463088e+00
index = 130 || gpu_result = 0.0000000000000000e+00 || cpu_result 4.3228554908580303e+00
index = 131 || gpu_result = 0.0000000000000000e+00 || cpu_result 3.1419158643118554e+00
index = 132 || gpu_result = 0.0000000000000000e+00 || cpu_result 2.3941805335452302e+00
index = 133 || gpu_result = 0.0000000000000000e+00 || cpu_result 1.7552125297690635e-01
index = 134 || gpu_result = 0.0000000000000000e+00 || cpu_result 4.3938594835621370e+00
index = 135 || gpu_result = 0.0000000000000000e+00 || cpu_result 2.2157291526343114e-02
index = 136 || gpu_result = 0.0000000000000000e+00 || cpu_result 5.4654162566210962e+00
index = 137 || gpu_result = 0.0000000000000000e+00 || cpu_result 4.5854895932974786e+00
index = 138 || gpu_result = 0.0000000000000000e+00 || cpu_result 1.1667387652029497e+00
index = 139 || gpu_result = 0.0000000000000000e+00 || cpu_result 5.4442468098008812e+00
index = 140 || gpu_result = 0.0000000000000000e+00 || cpu_result 1.9556984969700293e+00
index = 141 || gpu_result = 0.0000000000000000e+00 || cpu_result 4.0321519139284359e+00
index = 142 || gpu_result = 0.0000000000000000e+00 || cpu_result 4.9939591212880980e+00
index = 143 || gpu_result = 0.0000000000000000e+00 || cpu_result 4.4642884451117526e+00
index = 144 || gpu_result = 0.0000000000000000e+00 || cpu_result 3.4779853227861635e+00
index = 145 || gpu_result = 0.0000000000000000e+00 || cpu_result 3.9012637644093813e+00
index = 146 || gpu_result = 0.0000000000000000e+00 || cpu_result 3.2558029645346096e+00
index = 147 || gpu_result = 0.0000000000000000e+00 || cpu_result 5.8179570153596449e+00
index = 148 || gpu_result = 0.0000000000000000e+00 || cpu_result 1.8718053526492153e+00
index = 255 || gpu_result = 7.6375579412949691e+01 || cpu_result 5.5455861408909402e+02
TEST FAILED!

RESULTS ON AMD RX7900 GRE:

snarayanan@amd-pegasus:tests$ ./spmv
Running on Radeon RX 7900 GRE
TEST PASSED!
snarayanan@amd-pegasus:tests$ ./spmv
Running on Radeon RX 7900 GRE
RESULT DIFF DETECTED AT BELOW INDICES:
index = 156 || gpu_result = 2.3818103619925416e-102 || cpu_result 1.3906133564549183e+00
index = 157 || gpu_result = 2.5859252039660815e+306 || cpu_result 2.1163489249219523e+01
TEST FAILED!
snarayanan@amd-pegasus:tests$ ./spmv
Running on Radeon RX 7900 GRE
TEST PASSED!

oneAPI version: 2025.0.4

CODE : spmv.txt

COMPILE COMMAND:

icpx -fsycl -fsycl-targets=intel_gpu_pvc,amd_gpu_gfx1100,nvidia_gpu_sm_75 -I$ONEMATH_INC -L$ONEMATH_LIBS -o spmv -O3 spmv.cpp -lonemath 

NOTE:
Running it on a CPU deivce always seems to give the correct answer.

snarayanan@amd-pegasus:tests$ ONEAPI_DEVICE_SELECTOR=opencl:cpu ./spmv
Running on AMD Ryzen Threadripper 3970X 32-Core Processor
TEST PASSED!
snarayanan@amd-pegasus:tests$ ONEAPI_DEVICE_SELECTOR=opencl:cpu ./spmv
Running on AMD Ryzen Threadripper 3970X 32-Core Processor
TEST PASSED!
snarayanan@amd-pegasus:tests$ ONEAPI_DEVICE_SELECTOR=opencl:cpu ./spmv
Running on AMD Ryzen Threadripper 3970X 32-Core Processor
TEST PASSED!
snarayanan@amd-pegasus:tests$ ONEAPI_DEVICE_SELECTOR=opencl:cpu ./spmv
Running on AMD Ryzen Threadripper 3970X 32-Core Processor
TEST PASSED!
snarayanan@amd-pegasus:tests$ ONEAPI_DEVICE_SELECTOR=opencl:cpu ./spmv
Running on AMD Ryzen Threadripper 3970X 32-Core Processor
TEST PASSED!
snarayanan@amd-pegasus:tests$ ONEAPI_DEVICE_SELECTOR=opencl:cpu ./spmv
Running on AMD Ryzen Threadripper 3970X 32-Core Processor
TEST PASSED!
snarayanan@amd-pegasus:tests$ ONEAPI_DEVICE_SELECTOR=opencl:cpu ./spmv
Running on AMD Ryzen Threadripper 3970X 32-Core Processor
TEST PASSED!

Regards,
Sidarth Narayanan
[email protected]
Software Engineer - HPC Group

[sidarth-narayanan-csi]

Sorry all, I found the error. It was with my matrix generation function. Here is the updated version of the test if anyone is interested
CODE : spmv.cpp