From f1413a5df3ae7678b8d789466e4ca65b886d7008 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 31 Jul 2024 12:46:03 +0200 Subject: [PATCH 01/35] Bump sphinxcontrib-serializinghtml from 1.1.10 to 2.0.0 (#811) Bumps [sphinxcontrib-serializinghtml](https://github.com/sphinx-doc/sphinxcontrib-serializinghtml) from 1.1.10 to 2.0.0. - [Release notes](https://github.com/sphinx-doc/sphinxcontrib-serializinghtml/releases) - [Changelog](https://github.com/sphinx-doc/sphinxcontrib-serializinghtml/blob/master/CHANGES.rst) - [Commits](https://github.com/sphinx-doc/sphinxcontrib-serializinghtml/compare/1.1.10...2.0.0) --- updated-dependencies: - dependency-name: sphinxcontrib-serializinghtml dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index be22dbae69..02a39000ec 100644 --- a/requirements.txt +++ b/requirements.txt @@ -42,7 +42,7 @@ sphinxcontrib-devhelp==1.0.6 sphinxcontrib-htmlhelp==2.0.5 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.7 -sphinxcontrib-serializinghtml==1.1.10 +sphinxcontrib-serializinghtml==2.0.0 sphinxcontrib-spelling==8.0.0 typing_extensions==4.12.2 urllib3==2.2.2 From 06037deee626332a1f3ba0dc8e7437af70c41791 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 31 Jul 2024 13:03:39 +0200 Subject: [PATCH 02/35] Bump pre-commit from 3.7.1 to 3.8.0 (#807) Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.7.1 to 3.8.0. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](https://github.com/pre-commit/pre-commit/compare/v3.7.1...v3.8.0) --- updated-dependencies: - dependency-name: pre-commit dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 02a39000ec..ef1086f7df 100644 --- a/requirements.txt +++ b/requirements.txt @@ -25,7 +25,7 @@ MarkupSafe==2.1.5 nodeenv==1.9.1 packaging==24.1 platformdirs==4.2.2 -pre-commit==3.7.1 +pre-commit==3.8.0 pydata-sphinx-theme==0.15.3 pyenchant==3.2.2 Pygments==2.18.0 From 5c536b9d92e19a371f822ad30cc3be1bb323ce80 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 31 Jul 2024 13:26:34 +0200 Subject: [PATCH 03/35] Bump sphinxcontrib-htmlhelp from 2.0.5 to 2.1.0 (#809) Bumps [sphinxcontrib-htmlhelp](https://github.com/sphinx-doc/sphinxcontrib-htmlhelp) from 2.0.5 to 2.1.0. - [Release notes](https://github.com/sphinx-doc/sphinxcontrib-htmlhelp/releases) - [Changelog](https://github.com/sphinx-doc/sphinxcontrib-htmlhelp/blob/master/CHANGES.rst) - [Commits](https://github.com/sphinx-doc/sphinxcontrib-htmlhelp/compare/2.0.5...2.1.0) --- updated-dependencies: - dependency-name: sphinxcontrib-htmlhelp dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index ef1086f7df..ee9f81c067 100644 --- a/requirements.txt +++ b/requirements.txt @@ -39,7 +39,7 @@ Sphinx==7.4.7 sphinx-book-theme==1.1.3 sphinxcontrib-applehelp==1.0.8 sphinxcontrib-devhelp==1.0.6 -sphinxcontrib-htmlhelp==2.0.5 +sphinxcontrib-htmlhelp==2.1.0 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.7 sphinxcontrib-serializinghtml==2.0.0 From f25840ac92ff058eb3868e2fc09dd7f0d911f2d2 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 31 Jul 2024 13:43:32 +0200 Subject: [PATCH 04/35] Bump sphinxcontrib-devhelp from 1.0.6 to 2.0.0 (#810) Bumps [sphinxcontrib-devhelp](https://github.com/sphinx-doc/sphinxcontrib-devhelp) from 1.0.6 to 2.0.0. - [Release notes](https://github.com/sphinx-doc/sphinxcontrib-devhelp/releases) - [Changelog](https://github.com/sphinx-doc/sphinxcontrib-devhelp/blob/master/CHANGES.rst) - [Commits](https://github.com/sphinx-doc/sphinxcontrib-devhelp/compare/1.0.6...2.0.0) --- updated-dependencies: - dependency-name: sphinxcontrib-devhelp dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index ee9f81c067..bb7db65742 100644 --- a/requirements.txt +++ b/requirements.txt @@ -38,7 +38,7 @@ soupsieve==2.5 Sphinx==7.4.7 sphinx-book-theme==1.1.3 sphinxcontrib-applehelp==1.0.8 -sphinxcontrib-devhelp==1.0.6 +sphinxcontrib-devhelp==2.0.0 sphinxcontrib-htmlhelp==2.1.0 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.7 From d06cced896108f731430c5612ae96f97c04db60d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 31 Jul 2024 15:12:09 +0200 Subject: [PATCH 05/35] Bump reuse from 3.0.2 to 4.0.3 (#808) Bumps [reuse](https://github.com/fsfe/reuse-tool) from 3.0.2 to 4.0.3. - [Release notes](https://github.com/fsfe/reuse-tool/releases) - [Changelog](https://github.com/fsfe/reuse-tool/blob/main/CHANGELOG.md) - [Commits](https://github.com/fsfe/reuse-tool/compare/v3.0.2...v4.0.3) --- updated-dependencies: - dependency-name: reuse dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index bb7db65742..235c1f0532 100644 --- a/requirements.txt +++ b/requirements.txt @@ -32,7 +32,7 @@ Pygments==2.18.0 python-debian==0.1.49 PyYAML==6.0.1 requests==2.32.3 -reuse==3.0.2 +reuse==4.0.3 snowballstemmer==2.2.0 soupsieve==2.5 Sphinx==7.4.7 From 2879a3e44cdf08f67b8e1a6e6279fc7c16c78180 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20=C5=9Alusarczyk?= <112692748+lslusarczyk@users.noreply.github.com> Date: Wed, 7 Aug 2024 08:50:14 +0200 Subject: [PATCH 06/35] fix CI for oneDPL (#820) --- .github/workflows/onedpl.yml | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/.github/workflows/onedpl.yml b/.github/workflows/onedpl.yml index d1f1e8ae6e..3663f37f05 100644 --- a/.github/workflows/onedpl.yml +++ b/.github/workflows/onedpl.yml @@ -27,17 +27,11 @@ jobs: ref: distributed-ranges path: dr - name: Generate - run: | - . /opt/intel/oneapi/setvars.sh - cmake -B dr/build -DCMAKE_CXX_STANDARD=23 -DCMAKE_CXX_COMPILER=icpx -DONEDPL_BACKEND=dpcpp -DCMAKE_BUILD_TYPE=Release dr - - name: Build SP tests - run: | - . /opt/intel/oneapi/setvars.sh - cmake --build dr/build --target sp-all-tests -- -j + run: cmake -B dr/build -DCMAKE_CXX_STANDARD=23 -DCMAKE_CXX_COMPILER=icpx -DONEDPL_BACKEND=dpcpp -DCMAKE_BUILD_TYPE=Release dr + - name: Build SHP tests + run: cmake --build dr/build --target sp-all-tests -- -j - name: Run SP tests - run: | - . /opt/intel/oneapi/setvars.sh - ctest --test-dir dr/build -L SP -j 4 + run: ctest --test-dir dr/build -L SP -j 4 # srun -p cluster dr/scripts/run_command_on_compute_node.sh dr/build/Testing/tests.outerr.txt ctest --test-dir dr/build -L TESTLABEL -j 4 - uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4 if: always() From aa318b610db8bf699c6e8d872c8cd32689f2c02b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 14 Aug 2024 17:03:27 +0200 Subject: [PATCH 07/35] Bump pydata-sphinx-theme from 0.15.3 to 0.15.4 (#817) Bumps [pydata-sphinx-theme](https://github.com/pydata/pydata-sphinx-theme) from 0.15.3 to 0.15.4. - [Release notes](https://github.com/pydata/pydata-sphinx-theme/releases) - [Changelog](https://github.com/pydata/pydata-sphinx-theme/blob/main/RELEASE.md) - [Commits](https://github.com/pydata/pydata-sphinx-theme/compare/v0.15.3...v0.15.4) --- updated-dependencies: - dependency-name: pydata-sphinx-theme dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 235c1f0532..edb97f5a3c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -26,7 +26,7 @@ nodeenv==1.9.1 packaging==24.1 platformdirs==4.2.2 pre-commit==3.8.0 -pydata-sphinx-theme==0.15.3 +pydata-sphinx-theme==0.15.4 pyenchant==3.2.2 Pygments==2.18.0 python-debian==0.1.49 From ec3070424dd6248d5d2fcee71d47721cdf885765 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 16 Aug 2024 10:12:42 +0200 Subject: [PATCH 08/35] Bump actions/upload-artifact from 4.3.4 to 4.3.6 (#821) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.4 to 4.3.6. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/0b2256b8c012f0828dc542b3febcab082c67f72b...834a144ee995460fba8ed112a2fc961b36a5ec5a) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/onedpl.yml | 2 +- .github/workflows/pr.yml | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/onedpl.yml b/.github/workflows/onedpl.yml index 3663f37f05..15bab0dca1 100644 --- a/.github/workflows/onedpl.yml +++ b/.github/workflows/onedpl.yml @@ -33,7 +33,7 @@ jobs: - name: Run SP tests run: ctest --test-dir dr/build -L SP -j 4 # srun -p cluster dr/scripts/run_command_on_compute_node.sh dr/build/Testing/tests.outerr.txt ctest --test-dir dr/build -L TESTLABEL -j 4 - - uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4 + - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 if: always() with: name: log-pvc-impi-icpx diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index ab440d31c5..13af994c3e 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -55,7 +55,7 @@ jobs: # run: make -C build -j # - name: Unit tests # run: ctest --test-dir build -L TESTLABEL -j 4 -# - uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4 +# - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 # if: always() # with: # name: log-gcc-${{ env.CXX }} @@ -94,7 +94,7 @@ jobs: - name: Unit tests # run: srun -p cluster scripts/run_command_on_compute_node.sh build/Testing/tests.outerr.txt ctest --test-dir build -L TESTLABEL -j 4 run: ctest --test-dir build -L TESTLABEL -j 4 - - uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4 + - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 if: always() with: name: log-pvc-impi-icpx-${{ matrix.config }} @@ -135,7 +135,7 @@ jobs: # - name: MP unit tests # # run: srun -p cluster scripts/run_command_on_compute_node.sh build/Testing/mptests.outerr.txt ctest --test-dir build -R ^mp-tests-sycl -L MP -j 4 # run: ctest --test-dir build -R ^mp-tests-sycl -L MP -j 4 -# - uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4 +# - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 # if: always() # with: # name: log-ishmem-impi-icpx-${{ matrix.config }} From e60d69411ae5fed7d355e83445c9651eaa11841d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20=C5=9Alusarczyk?= <112692748+lslusarczyk@users.noreply.github.com> Date: Wed, 28 Aug 2024 10:00:48 +0200 Subject: [PATCH 09/35] running CI on CPU and public runners (#822) * running CI on CPU and public runners --------- Co-authored-by: Mikolaj Komar --- .github/workflows/pr.yml | 37 +++++++++++++++++++++-------- CMakeLists.txt | 14 ++++++++--- benchmarks/gbench/sp/CMakeLists.txt | 2 +- include/CMakeLists.txt | 2 +- test/gtest/mp/CMakeLists.txt | 13 +++++----- 5 files changed, 47 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 13af994c3e..d1bca696d0 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -66,8 +66,9 @@ jobs: # build/examples/mp/*.log pvc_unit_tests: - runs-on: gkpvc - timeout-minutes: 30 + # runs-on: gkpvc + runs-on: ubuntu-24.04 + timeout-minutes: 120 strategy: # test everything, even if one fails fail-fast: false @@ -82,18 +83,34 @@ jobs: FI_PROVIDER: tcp steps: - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 + - name: Set up Intel APT repository + run: | + # https://www.intel.com/content/www/us/en/docs/oneapi/installation-guide-linux/2024-0/apt.html + wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \ + | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null + echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list + sudo apt-get update -y + - name: Install oneAPI + run: sudo apt-get install intel-oneapi-compiler-dpcpp-cpp intel-oneapi-mkl-devel intel-hpckit -y - name: Generate - run: cmake -B build -DCMAKE_BUILD_TYPE=${{ matrix.config }} - - name: Save environment dump - run: printenv > build/envdump.txt + run: | + . /opt/intel/oneapi/setvars.sh + cmake -B build -DCMAKE_BUILD_TYPE=${{ matrix.config }} + printenv > build/envdump.txt # change this step to building just tests once unittest job is back - - name: Build everything - run: make -C build -j -# - name: Build tests -# run: cmake --build build --target all-tests -- -j +# - name: Build everything +# run: | +# . /opt/intel/oneapi/setvars.sh +# make -C build -j + - name: Build tests + run: | + . /opt/intel/oneapi/setvars.sh + cmake --build build --target all-tests -- -j 4 - name: Unit tests # run: srun -p cluster scripts/run_command_on_compute_node.sh build/Testing/tests.outerr.txt ctest --test-dir build -L TESTLABEL -j 4 - run: ctest --test-dir build -L TESTLABEL -j 4 + run: | + . /opt/intel/oneapi/setvars.sh + ctest --test-dir build --output-on-failure -L TESTLABEL -j 4 - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 if: always() with: diff --git a/CMakeLists.txt b/CMakeLists.txt index 3ce7a0d6c5..8b963e0290 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -153,7 +153,7 @@ add_custom_target(sp-all-tests) add_custom_target(all-tests DEPENDS mp-all-tests sp-all-tests) function(add_mp_ctest_impl) - set(options OFFLOAD GDB SYCL DRLOGS TESTLABEL) + set(options OFFLOAD GDB SYCL DRLOGS TESTLABEL PVCLABEL) set(oneValueArgs NAME TEST_NAME NPROC TIMEOUT) set(multiValueArgs TARGS) # Test ARGumentS @@ -234,7 +234,10 @@ function(add_mp_ctest_impl) endif() if(AMC_TESTLABEL) - set_property(TEST ${AMC_TEST_NAME} PROPERTY LABELS TESTLABEL MP) + set_property(TEST ${AMC_TEST_NAME} PROPERTY LABELS TESTLABEL PVCLABEL MP) + endif() + if(AMC_PVCLABEL) + set_property(TEST ${AMC_TEST_NAME} PROPERTY LABELS PVCLABEL MP) endif() add_dependencies(mp-all-tests ${AMC_NAME}) endfunction() @@ -250,6 +253,11 @@ function(add_mp_ctest) add_mp_auxiliary_ctests(${ARGN}) endfunction() +function(add_mp_pvc_ctest) + add_mp_ctest_impl(${ARGN} PVCLABEL) + add_mp_auxiliary_ctests(${ARGN}) +endfunction() + function(add_mp_disabled_ctest) add_mp_ctest_impl(${ARGN}) add_mp_auxiliary_ctests(${ARGN}) @@ -370,7 +378,7 @@ endfunction() function(add_sp_ctest test_name name) add_sp_disabled_ctest(${test_name} ${name} ${ARGN}) - set_property(TEST ${test_name} PROPERTY LABELS TESTLABEL SP) + set_property(TEST ${test_name} PROPERTY LABELS TESTLABEL PVCLABEL SP) endfunction() install(DIRECTORY include DESTINATION ${CMAKE_INSTALL_PREFIX}) diff --git a/benchmarks/gbench/sp/CMakeLists.txt b/benchmarks/gbench/sp/CMakeLists.txt index c1551ef47e..7b1ba01bc7 100644 --- a/benchmarks/gbench/sp/CMakeLists.txt +++ b/benchmarks/gbench/sp/CMakeLists.txt @@ -25,7 +25,7 @@ add_executable( gemm.cpp ../common/distributed_vector.cpp ../common/dot_product.cpp - ../common/inclusive_exclusive_scan.cpp +# ../common/inclusive_exclusive_scan.cpp disabled due to deadlock DRA-214 ../common/sort.cpp ../common/stream.cpp) # cmake-format: on diff --git a/include/CMakeLists.txt b/include/CMakeLists.txt index 862b0bb138..5176e13e8c 100644 --- a/include/CMakeLists.txt +++ b/include/CMakeLists.txt @@ -5,7 +5,7 @@ FetchContent_Declare( dpl GIT_REPOSITORY https://github.com/oneapi-src/oneDPL.git - GIT_TAG 251b0b1) + GIT_TAG main) FetchContent_MakeAvailable(dpl) # mpi: multi-process diff --git a/test/gtest/mp/CMakeLists.txt b/test/gtest/mp/CMakeLists.txt index cef65af431..57bbaf0d5f 100644 --- a/test/gtest/mp/CMakeLists.txt +++ b/test/gtest/mp/CMakeLists.txt @@ -16,10 +16,10 @@ add_executable( ../common/drop.cpp ../common/enumerate.cpp ../common/equal.cpp - ../common/exclusive_scan.cpp +# ../common/exclusive_scan.cpp disabled due to deadlock - DRA-213 ../common/fill.cpp ../common/for_each.cpp - ../common/inclusive_scan.cpp +# ../common/inclusive_scan.cpp disabled due to deadlock - DRA-213 ../common/iota.cpp ../common/iota_view.cpp ../common/reduce.cpp @@ -107,16 +107,17 @@ if(ENABLE_SYCL) foreach(nproc RANGE 1 4) add_mp_ctest(NAME mp-quick-test NPROC ${nproc} SYCL) - add_mp_ctest( + add_mp_pvc_ctest( NAME mp-quick-test NPROC ${nproc} OFFLOAD SYCL TARGS --device-memory) endforeach() - add_mp_ctest( + add_mp_pvc_ctest( NAME mp-tests NPROC 2 TIMEOUT 150 OFFLOAD SYCL TARGS --device-memory --gtest_filter=-${sycl-exclusions}) if(NOT MPI_IMPL STREQUAL "openmpi") - # MPI_Win_create fails for communicator with size 1 - add_mp_ctest( + # MPI_Win_create fails for communicator with size 1 this should be + # add_mp_ctest - DRA-215 + add_mp_pvc_ctest( NAME mp-tests NPROC 1 SYCL TARGS --gtest_filter=-${sycl-exclusions}) endif() # TODO: fix sycl Slide issues, see From 10f8b32a1eba6bb76d4ab2cd44cee22d8edd13ca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Komar?= <69756491+Xewar313@users.noreply.github.com> Date: Wed, 28 Aug 2024 10:35:32 +0200 Subject: [PATCH 10/35] Add tests and fixes to sp sparse matrix (#819) --- include/dr/detail/index.hpp | 11 ++ include/dr/sp/algorithms/matrix/gemv.hpp | 5 +- include/dr/sp/containers/matrix_partition.hpp | 7 +- include/dr/sp/containers/sparse_matrix.hpp | 24 ++- include/dr/sp/util/matrix_io.hpp | 6 +- include/dr/sp/views/csr_matrix_view.hpp | 4 +- test/gtest/sp/CMakeLists.txt | 4 +- test/gtest/sp/gemv.cpp | 149 ++++++++++++++++++ test/gtest/sp/sparse.cpp | 69 ++++++++ 9 files changed, 262 insertions(+), 17 deletions(-) create mode 100644 test/gtest/sp/sparse.cpp diff --git a/include/dr/detail/index.hpp b/include/dr/detail/index.hpp index a317e97bcf..3c496e8a57 100644 --- a/include/dr/detail/index.hpp +++ b/include/dr/detail/index.hpp @@ -72,6 +72,17 @@ template class index { } constexpr bool operator==(const index &) const noexcept = default; + constexpr bool operator<(const index &other) const noexcept + requires(std::totally_ordered) + { + if (first < other.first) { + return true; + } + if (first == other.first && second < other.second) { + return true; + } + return false; + } template constexpr T get() const noexcept diff --git a/include/dr/sp/algorithms/matrix/gemv.hpp b/include/dr/sp/algorithms/matrix/gemv.hpp index bb813cb61e..98318552ed 100644 --- a/include/dr/sp/algorithms/matrix/gemv.hpp +++ b/include/dr/sp/algorithms/matrix/gemv.hpp @@ -19,7 +19,7 @@ namespace dr::sp { template void flat_gemv(C &&c, dr::sp::sparse_matrix &a, B &&b) { - assert(c.size() == b.size()); + assert(a.shape()[0] == c.size()); assert(a.shape()[1] == b.size()); assert(a.grid_shape()[0] == c.segments().size()); assert(a.grid_shape()[1] == 1); @@ -79,7 +79,7 @@ template void gemv(C &&c, dr::sp::sparse_matrix &a, B &&b, sp::duplicated_vector> &scratch) { - assert(c.size() == b.size()); + assert(a.shape()[0] == c.size()); assert(a.shape()[1] == b.size()); assert(a.grid_shape()[0] == c.segments().size()); assert(a.grid_shape()[1] == 1); @@ -104,6 +104,7 @@ void gemv(C &&c, dr::sp::sparse_matrix &a, B &&b, dr::ranges::local(b_duplicated.local_vector(a_tile.rank()).begin()); auto c_iter = dr::ranges::local(c.segments()[i].begin()); + assert(c.segments()[i].size() == a_tile.shape()[0]); auto &&q = __detail::queue(a_tile.rank()); auto event = __detail::local_gemv(q, a_tile, b_iter, c_iter, diff --git a/include/dr/sp/containers/matrix_partition.hpp b/include/dr/sp/containers/matrix_partition.hpp index 358a747f69..631835d9d8 100644 --- a/include/dr/sp/containers/matrix_partition.hpp +++ b/include/dr/sp/containers/matrix_partition.hpp @@ -86,7 +86,12 @@ class block_cyclic final : public matrix_partition { dr::index<> tile_shape_; dr::index<> grid_shape_; -}; // namespace dr::sp +}; + +inline auto row_cyclic() { + return block_cyclic({dr::sp::tile::div, dr::sp::tile::div}, + {dr::sp::nprocs(), 1}); +} inline std::vector partition_matmul(std::size_t m, std::size_t n, std::size_t k) { diff --git a/include/dr/sp/containers/sparse_matrix.hpp b/include/dr/sp/containers/sparse_matrix.hpp index 96746d8398..a8184633cb 100644 --- a/include/dr/sp/containers/sparse_matrix.hpp +++ b/include/dr/sp/containers/sparse_matrix.hpp @@ -46,7 +46,13 @@ class distributed_range_accessor { constexpr distributed_range_accessor(Segments segments, size_type segment_id, size_type idx) noexcept : segments_(rng::views::all(std::forward(segments))), - segment_id_(segment_id), idx_(idx) {} + segment_id_(segment_id), idx_(idx) { + while (idx_ >= rng::size((*(segments_.begin() + segment_id_))) && + segment_id_ < rng::size(segments_)) { + segment_id_++; + idx_ = 0; + } + } constexpr distributed_range_accessor & operator+=(difference_type offset) noexcept { @@ -59,7 +65,8 @@ class distributed_range_accessor { idx_ += current_offset; offset -= current_offset; - if (idx_ >= rng::size((*(segments_.begin() + segment_id_)))) { + while (idx_ >= rng::size((*(segments_.begin() + segment_id_))) && + segment_id_ < rng::size(segments_)) { segment_id_++; idx_ = 0; } @@ -71,14 +78,16 @@ class distributed_range_accessor { difference_type new_idx = difference_type(idx_) - current_offset; - if (new_idx < 0) { + while (new_idx < 0 && segment_id_ > 0) { segment_id_--; new_idx = rng::size(*(segments_.begin() + segment_id_)) - 1; } idx_ = new_idx; + offset += current_offset; } + assert(offset == 0); return *this; } @@ -151,12 +160,12 @@ template class sparse_matrix { distributed_sparse_matrix_iterator &&>; sparse_matrix(key_type shape) - : shape_(shape), partition_(new dr::sp::block_cyclic()) { + : shape_(shape), partition_(default_partition_()) { init_(); } sparse_matrix(key_type shape, double density) - : shape_(shape), partition_(new dr::sp::block_cyclic()) { + : shape_(shape), partition_(default_partition_()) { init_random_(density); } @@ -393,6 +402,11 @@ template class sparse_matrix { segments_ = generate_segments_(); } + std::unique_ptr default_partition_() { + auto ptr = dr::sp::row_cyclic(); + return std::make_unique(ptr); + } + private: key_type shape_; key_type grid_shape_; diff --git a/include/dr/sp/util/matrix_io.hpp b/include/dr/sp/util/matrix_io.hpp index d0403535e3..d34fd132f7 100644 --- a/include/dr/sp/util/matrix_io.hpp +++ b/include/dr/sp/util/matrix_io.hpp @@ -279,11 +279,7 @@ auto mmread(std::string file_path, const matrix_partition &partition, template auto mmread(std::string file_path, bool one_indexed = true) { - return mmread( - file_path, - dr::sp::block_cyclic({dr::sp::tile::div, dr::sp::tile::div}, - {dr::sp::nprocs(), 1}), - one_indexed); + return mmread(file_path, dr::sp::row_cyclic(), one_indexed); } } // namespace dr::sp diff --git a/include/dr/sp/views/csr_matrix_view.hpp b/include/dr/sp/views/csr_matrix_view.hpp index d0705175be..c698cc9c9d 100644 --- a/include/dr/sp/views/csr_matrix_view.hpp +++ b/include/dr/sp/views/csr_matrix_view.hpp @@ -156,11 +156,11 @@ class csr_matrix_view std::size_t rank() const { return rank_; } iterator begin() const { - return iterator(values_, rowptr_, colind_, 0, 0, shape()[1], idx_offset_); + return iterator(values_, rowptr_, colind_, 0, 0, shape()[0], idx_offset_); } iterator end() const { - return iterator(values_, rowptr_, colind_, nnz_, shape()[1], shape()[1], + return iterator(values_, rowptr_, colind_, nnz_, shape()[0], shape()[0], idx_offset_); } diff --git a/test/gtest/sp/CMakeLists.txt b/test/gtest/sp/CMakeLists.txt index 6470eeee19..474103f0da 100644 --- a/test/gtest/sp/CMakeLists.txt +++ b/test/gtest/sp/CMakeLists.txt @@ -14,12 +14,12 @@ add_executable( ../common/reduce.cpp ../common/sort.cpp ../common/subrange.cpp ../common/take.cpp ../common/transform.cpp ../common/transform_view.cpp ../common/zip.cpp ../common/zip_local.cpp containers.cpp algorithms.cpp - copy.cpp detail.cpp fill.cpp gemv.cpp transform.cpp) + copy.cpp detail.cpp fill.cpp gemv.cpp sparse.cpp transform.cpp) add_executable(sp-tests-3 sp-tests.cpp containers-3.cpp copy-3.cpp) # skeleton for rapid builds of individual tests, feel free to change this -add_executable(sp-quick-test sp-tests.cpp ../common/equal.cpp) +add_executable(sp-quick-test sp-tests.cpp sparse.cpp) target_compile_definitions(sp-quick-test PRIVATE QUICK_TEST) foreach(test-exec IN ITEMS sp-tests sp-tests-3 sp-quick-test) diff --git a/test/gtest/sp/gemv.cpp b/test/gtest/sp/gemv.cpp index d7807448a8..27a1cc40ec 100644 --- a/test/gtest/sp/gemv.cpp +++ b/test/gtest/sp/gemv.cpp @@ -33,3 +33,152 @@ TEST(SparseMatrix, Gemv) { EXPECT_TRUE(fp_equal(c_ref, c_local)) << fmt::format("Reference:\n {}\nActual:\n {}\n", c_ref, c_local); } + +TEST(SparseMatrix, EmptyGemv) { + std::size_t m = 100; + std::size_t k = 100; + using T = float; + using I = int; + + dr::sp::__detail::coo_matrix base; + auto csr = dr::sp::__detail::convert_to_csr(base, {m, k}, base.size(), + std::allocator{}); + dr::sp::sparse_matrix a = + dr::sp::create_distributed(csr, dr::sp::row_cyclic()); + + dr::sp::distributed_vector b(k, 1.f); + dr::sp::distributed_vector c(m, 0.f); + + dr::sp::gemv(c, a, b); + + std::vector c_local(m); + + dr::sp::copy(c.begin(), c.end(), c_local.begin()); + + std::vector c_ref(m, 0.f); + + EXPECT_TRUE(fp_equal(c_ref, c_local)) + << fmt::format("Reference:\n {}\nActual:\n {}\n", c_ref, c_local); +} + +TEST(SparseMatrix, ZeroVector) { + std::size_t m = 100; + std::size_t k = 100; + using T = float; + using I = int; + std::vector, T>> base; + for (int i = 0; i < m; i++) { + for (int j = 0; j < k; j++) { + base.push_back({{i, j}, i + j}); + } + } + + auto csr = dr::sp::__detail::convert_to_csr(base, {m, k}, base.size(), + std::allocator{}); + dr::sp::sparse_matrix a = + dr::sp::create_distributed(csr, dr::sp::row_cyclic()); + + dr::sp::distributed_vector b(k, 0.f); + dr::sp::distributed_vector c(m, 0.f); + + dr::sp::gemv(c, a, b); + + std::vector c_local(m); + + dr::sp::copy(c.begin(), c.end(), c_local.begin()); + + std::vector c_ref(m, 0.f); + + EXPECT_TRUE(fp_equal(c_ref, c_local)) + << fmt::format("Reference:\n {}\nActual:\n {}\n", c_ref, c_local); +} + +TEST(SparseMatrix, NotSquareMatrix) { + std::size_t m = 10; + std::size_t k = 1000; + + dr::sp::sparse_matrix a( + {m, k}, 0.1f, + dr::sp::block_cyclic({dr::sp::tile::div, dr::sp::tile::div}, + {dr::sp::nprocs(), 1})); + + dr::sp::distributed_vector b(k, 1.f); + dr::sp::distributed_vector c(m, 0.f); + + dr::sp::gemv(c, a, b); + + std::vector c_local(m); + + dr::sp::copy(c.begin(), c.end(), c_local.begin()); + + std::vector c_ref(m, 0.f); + + for (auto &&[index, v] : a) { + auto &&[i, k] = index; + + c_ref[i] += v; + } + + EXPECT_TRUE(fp_equal(c_ref, c_local)) + << fmt::format("Reference:\n {}\nActual:\n {}\n", c_ref, c_local); +} + +TEST(SparseMatrix, NotSquareMatrixOtherAxis) { + std::size_t m = 1000; + std::size_t k = 10; + + dr::sp::sparse_matrix a( + {m, k}, 0.1f, + dr::sp::block_cyclic({dr::sp::tile::div, dr::sp::tile::div}, + {dr::sp::nprocs(), 1})); + + dr::sp::distributed_vector b(k, 1.f); + dr::sp::distributed_vector c(m, 0.f); + + dr::sp::gemv(c, a, b); + + std::vector c_local(m); + + dr::sp::copy(c.begin(), c.end(), c_local.begin()); + + std::vector c_ref(m, 0.f); + + for (auto &&[index, v] : a) { + auto &&[i, k] = index; + + c_ref[i] += v; + } + + EXPECT_TRUE(fp_equal(c_ref, c_local)) + << fmt::format("Reference:\n {}\nActual:\n {}\n", c_ref, c_local); +} + +TEST(SparseMatrix, VerySparseMatrix) { + std::size_t m = 100; + std::size_t k = 100; + + dr::sp::sparse_matrix a( + {m, k}, 0.001f, + dr::sp::block_cyclic({dr::sp::tile::div, dr::sp::tile::div}, + {dr::sp::nprocs(), 1})); + + dr::sp::distributed_vector b(k, 1.f); + dr::sp::distributed_vector c(m, 0.f); + + dr::sp::gemv(c, a, b); + + std::vector c_local(m); + + dr::sp::copy(c.begin(), c.end(), c_local.begin()); + + std::vector c_ref(m, 0.f); + + for (auto &&[index, v] : a) { + auto &&[i, k] = index; + + c_ref[i] += v; + } + + EXPECT_TRUE(fp_equal(c_ref, c_local)) + << fmt::format("Reference:\n {}\nActual:\n {}\n", c_ref, c_local); +} diff --git a/test/gtest/sp/sparse.cpp b/test/gtest/sp/sparse.cpp new file mode 100644 index 0000000000..7d660b7e75 --- /dev/null +++ b/test/gtest/sp/sparse.cpp @@ -0,0 +1,69 @@ +// SPDX-FileCopyrightText: Intel Corporation +// +// SPDX-License-Identifier: BSD-3-Clause +#include "xp-tests.hpp" + +TEST(SparseMatrix, IterationForward) { + std::size_t m = 10; + std::size_t k = 10; + using T = float; + using I = int; + std::vector, T>> base; + for (int i = 0; i < m; i++) { + for (int j = 0; j < k; j++) { + base.push_back({{i, j}, i + j}); + } + } + std::vector, T>> reference(base.size()); + std::copy(base.begin(), base.end(), reference.begin()); + auto csr = dr::sp::__detail::convert_to_csr(base, {m, k}, base.size(), + std::allocator{}); + dr::sp::sparse_matrix a = + dr::sp::create_distributed(csr, dr::sp::row_cyclic()); + int i = 0; + for (auto elem : a) { + auto [index, value] = elem; + auto [real_index, real_value] = reference[i]; + auto [m, n] = index; + auto [r_m, r_n] = real_index; + + EXPECT_TRUE(m == r_m && n == r_n) << fmt::format( + "Reference m, n:\n {}, {}\nActual:\n {}, {}\n", r_m, r_n, m, n); + EXPECT_TRUE(value == real_value) << fmt::format( + "Reference value:\n {}\nActual:\n {}\n", real_value, value); + i++; + } +} + +TEST(SparseMatrix, IterationReverse) { + std::size_t m = 10; + std::size_t k = 10; + using T = float; + using I = int; + std::vector, T>> base; + for (int i = 0; i < m; i++) { + for (int j = 0; j < k; j++) { + base.push_back({{i, j}, i + j}); + } + } + std::vector, T>> reference(base.size()); + std::copy(base.begin(), base.end(), reference.begin()); + auto csr = dr::sp::__detail::convert_to_csr(base, {m, k}, base.size(), + std::allocator{}); + dr::sp::sparse_matrix a = + dr::sp::create_distributed(csr, dr::sp::row_cyclic()); + int i = base.size(); + auto iterator = a.end(); + while (iterator > a.begin()) { + iterator--; + i--; + auto [index, value] = *iterator; + auto [real_index, real_value] = reference[i]; + auto [m, n] = index; + auto [r_m, r_n] = real_index; + EXPECT_TRUE(m == r_m && n == r_n) << fmt::format( + "Reference m, n:\n {}, {}\nActual:\n {}, {}\n", r_m, r_n, m, n); + EXPECT_TRUE(value == real_value) << fmt::format( + "Reference value:\n {}\nActual:\n {}\n", real_value, value); + } +} From 9329ff917f87a47af973754e1ffa5f8c45a7e66e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Komar?= <69756491+Xewar313@users.noreply.github.com> Date: Wed, 28 Aug 2024 15:26:25 +0200 Subject: [PATCH 11/35] Fix test casting error (#826) --- test/gtest/sp/gemv.cpp | 2 +- test/gtest/sp/sparse.cpp | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/test/gtest/sp/gemv.cpp b/test/gtest/sp/gemv.cpp index 27a1cc40ec..2b207be926 100644 --- a/test/gtest/sp/gemv.cpp +++ b/test/gtest/sp/gemv.cpp @@ -69,7 +69,7 @@ TEST(SparseMatrix, ZeroVector) { std::vector, T>> base; for (int i = 0; i < m; i++) { for (int j = 0; j < k; j++) { - base.push_back({{i, j}, i + j}); + base.push_back({{i, j}, static_cast(i + j)}); } } diff --git a/test/gtest/sp/sparse.cpp b/test/gtest/sp/sparse.cpp index 7d660b7e75..2e30fee2a0 100644 --- a/test/gtest/sp/sparse.cpp +++ b/test/gtest/sp/sparse.cpp @@ -11,7 +11,7 @@ TEST(SparseMatrix, IterationForward) { std::vector, T>> base; for (int i = 0; i < m; i++) { for (int j = 0; j < k; j++) { - base.push_back({{i, j}, i + j}); + base.push_back({{i, j}, static_cast(i + j)}); } } std::vector, T>> reference(base.size()); @@ -43,7 +43,7 @@ TEST(SparseMatrix, IterationReverse) { std::vector, T>> base; for (int i = 0; i < m; i++) { for (int j = 0; j < k; j++) { - base.push_back({{i, j}, i + j}); + base.push_back({{i, j}, static_cast(i + j)}); } } std::vector, T>> reference(base.size()); From 2a3c4d5308a9f3116ff39ab629b85176243c23d4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 28 Aug 2024 14:26:24 +0000 Subject: [PATCH 12/35] Bump babel from 2.15.0 to 2.16.0 (#824) Bumps [babel](https://github.com/python-babel/babel) from 2.15.0 to 2.16.0. - [Release notes](https://github.com/python-babel/babel/releases) - [Changelog](https://github.com/python-babel/babel/blob/master/CHANGES.rst) - [Commits](https://github.com/python-babel/babel/compare/v2.15.0...v2.16.0) --- updated-dependencies: - dependency-name: babel dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index edb97f5a3c..9291a5baa3 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,7 +4,7 @@ accessible-pygments==0.0.5 alabaster==0.7.16 -Babel==2.15.0 +Babel==2.16.0 beautifulsoup4==4.12.3 binaryornot==0.4.4 boolean.py==4.0 From 08b788fa7d25d8be5f39413be9eea0f3f5f49037 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 28 Aug 2024 14:26:35 +0000 Subject: [PATCH 13/35] Bump sphinxcontrib-qthelp from 1.0.7 to 2.0.0 (#816) Bumps [sphinxcontrib-qthelp](https://github.com/sphinx-doc/sphinxcontrib-qthelp) from 1.0.7 to 2.0.0. - [Release notes](https://github.com/sphinx-doc/sphinxcontrib-qthelp/releases) - [Changelog](https://github.com/sphinx-doc/sphinxcontrib-qthelp/blob/master/CHANGES.rst) - [Commits](https://github.com/sphinx-doc/sphinxcontrib-qthelp/compare/1.0.7...2.0.0) --- updated-dependencies: - dependency-name: sphinxcontrib-qthelp dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 9291a5baa3..510f201090 100644 --- a/requirements.txt +++ b/requirements.txt @@ -41,7 +41,7 @@ sphinxcontrib-applehelp==1.0.8 sphinxcontrib-devhelp==2.0.0 sphinxcontrib-htmlhelp==2.1.0 sphinxcontrib-jsmath==1.0.1 -sphinxcontrib-qthelp==1.0.7 +sphinxcontrib-qthelp==2.0.0 sphinxcontrib-serializinghtml==2.0.0 sphinxcontrib-spelling==8.0.0 typing_extensions==4.12.2 From fa134597af812c3c0cf4b1fea83bc5d1956c010a Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 28 Aug 2024 14:26:52 +0000 Subject: [PATCH 14/35] Bump sphinxcontrib-applehelp from 1.0.8 to 2.0.0 (#814) Bumps [sphinxcontrib-applehelp](https://github.com/sphinx-doc/sphinxcontrib-applehelp) from 1.0.8 to 2.0.0. - [Release notes](https://github.com/sphinx-doc/sphinxcontrib-applehelp/releases) - [Changelog](https://github.com/sphinx-doc/sphinxcontrib-applehelp/blob/master/CHANGES.rst) - [Commits](https://github.com/sphinx-doc/sphinxcontrib-applehelp/compare/1.0.8...2.0.0) --- updated-dependencies: - dependency-name: sphinxcontrib-applehelp dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 510f201090..81144ee540 100644 --- a/requirements.txt +++ b/requirements.txt @@ -37,7 +37,7 @@ snowballstemmer==2.2.0 soupsieve==2.5 Sphinx==7.4.7 sphinx-book-theme==1.1.3 -sphinxcontrib-applehelp==1.0.8 +sphinxcontrib-applehelp==2.0.0 sphinxcontrib-devhelp==2.0.0 sphinxcontrib-htmlhelp==2.1.0 sphinxcontrib-jsmath==1.0.1 From 7f86ed20e8aef37222ac7a1a230265b8a4fe2c4a Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 28 Aug 2024 18:27:37 +0200 Subject: [PATCH 15/35] Bump sphinx from 7.4.7 to 8.0.2 (#815) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 7.4.7 to 8.0.2. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/v8.0.2/CHANGES.rst) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v7.4.7...v8.0.2) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 81144ee540..2a9d57d883 100644 --- a/requirements.txt +++ b/requirements.txt @@ -35,7 +35,7 @@ requests==2.32.3 reuse==4.0.3 snowballstemmer==2.2.0 soupsieve==2.5 -Sphinx==7.4.7 +Sphinx==8.0.2 sphinx-book-theme==1.1.3 sphinxcontrib-applehelp==2.0.0 sphinxcontrib-devhelp==2.0.0 From 6ea15ed875883db83247beeefc33507f219c03d4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 29 Aug 2024 13:55:20 +0200 Subject: [PATCH 16/35] Bump alabaster from 0.7.16 to 1.0.0 (#813) Bumps [alabaster](https://github.com/sphinx-doc/alabaster) from 0.7.16 to 1.0.0. - [Release notes](https://github.com/sphinx-doc/alabaster/releases) - [Changelog](https://github.com/sphinx-doc/alabaster/blob/master/docs/changelog.rst) - [Commits](https://github.com/sphinx-doc/alabaster/compare/0.7.16...1.0.0) --- updated-dependencies: - dependency-name: alabaster dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 2a9d57d883..44474a6c1a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -3,7 +3,7 @@ # SPDX-License-Identifier: BSD-3-Clause accessible-pygments==0.0.5 -alabaster==0.7.16 +alabaster==1.0.0 Babel==2.16.0 beautifulsoup4==4.12.3 binaryornot==0.4.4 From f69a3c36aaa0bf28eb334b6af619051546781732 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20=C5=9Alusarczyk?= <112692748+lslusarczyk@users.noreply.github.com> Date: Thu, 29 Aug 2024 17:19:07 +0200 Subject: [PATCH 17/35] removed oneDPL CI (#782) not working and not needed --- .github/workflows/onedpl.yml | 42 ------------------------------------ 1 file changed, 42 deletions(-) delete mode 100644 .github/workflows/onedpl.yml diff --git a/.github/workflows/onedpl.yml b/.github/workflows/onedpl.yml deleted file mode 100644 index 15bab0dca1..0000000000 --- a/.github/workflows/onedpl.yml +++ /dev/null @@ -1,42 +0,0 @@ -# SPDX-FileCopyrightText: Intel Corporation -# -# SPDX-License-Identifier: BSD-3-Clause - -name: "oneDPL" -permissions: read-all - -on: - workflow_dispatch: - # 6am UTC is 10pm PST, 11pm PDT - schedule: - - cron: '0 6 * * *' - -jobs: - onedpl: - runs-on: gkpvc - timeout-minutes: 20 - env: - CXX: icpx - CTEST_OUTPUT_ON_FAILURE: 1 - steps: - - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 - - name: Checkout Distributed Ranges branch in oneDPL - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 - with: - repository: oneapi-src/oneDPL - ref: distributed-ranges - path: dr - - name: Generate - run: cmake -B dr/build -DCMAKE_CXX_STANDARD=23 -DCMAKE_CXX_COMPILER=icpx -DONEDPL_BACKEND=dpcpp -DCMAKE_BUILD_TYPE=Release dr - - name: Build SHP tests - run: cmake --build dr/build --target sp-all-tests -- -j - - name: Run SP tests - run: ctest --test-dir dr/build -L SP -j 4 - # srun -p cluster dr/scripts/run_command_on_compute_node.sh dr/build/Testing/tests.outerr.txt ctest --test-dir dr/build -L TESTLABEL -j 4 - - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 - if: always() - with: - name: log-pvc-impi-icpx - path: | - dr/build/CMakeCache.txt - dr/build/test/distributed-ranges/sp/*.log From b83de90d1842100722310035bd2ddd688312a40d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20=C5=9Alusarczyk?= <112692748+lslusarczyk@users.noreply.github.com> Date: Fri, 20 Sep 2024 09:46:33 +0200 Subject: [PATCH 18/35] increased timeout for multi proc tests (#836) --- test/gtest/mp/CMakeLists.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/gtest/mp/CMakeLists.txt b/test/gtest/mp/CMakeLists.txt index 57bbaf0d5f..8138b35fb0 100644 --- a/test/gtest/mp/CMakeLists.txt +++ b/test/gtest/mp/CMakeLists.txt @@ -125,10 +125,10 @@ if(ENABLE_SYCL) foreach(nproc RANGE 2 4) add_mp_ctest( - NAME mp-tests NPROC ${nproc} TIMEOUT 150 SYCL TARGS + NAME mp-tests NPROC ${nproc} TIMEOUT 500 SYCL TARGS --gtest_filter=-${sycl-exclusions}) endforeach() add_mp_ctest( - TEST_NAME mp-tests-sycl-3-only NAME mp-tests-3 NPROC 3 TIMEOUT 150 SYCL + TEST_NAME mp-tests-sycl-3-only NAME mp-tests-3 NPROC 3 TIMEOUT 500 SYCL TARGS --gtest_filter=-${sycl-exclusions}) endif() From 5967b583fd651e43f4f07c76967254a94c9db2de Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 20 Sep 2024 15:41:56 +0200 Subject: [PATCH 19/35] Bump idna from 3.7 to 3.10 (#835) Bumps [idna](https://github.com/kjd/idna) from 3.7 to 3.10. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](https://github.com/kjd/idna/compare/v3.7...v3.10) --- updated-dependencies: - dependency-name: idna dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 44474a6c1a..3be22e3826 100644 --- a/requirements.txt +++ b/requirements.txt @@ -17,7 +17,7 @@ distlib==0.3.8 docutils==0.21.2 filelock==3.15.4 identify==2.6.0 -idna==3.7 +idna==3.10 imagesize==1.4.1 Jinja2==3.1.4 license-expression==30.2.0 From f778c5847343b4228ff2750adfc2271e88168a00 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 20 Sep 2024 15:43:05 +0200 Subject: [PATCH 20/35] Bump actions/upload-artifact from 4.3.6 to 4.4.0 (#833) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.6 to 4.4.0. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/834a144ee995460fba8ed112a2fc961b36a5ec5a...50769540e7f4bd5e21e526ee35c689e35e0d6874) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/pr.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index d1bca696d0..f927e82972 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -55,7 +55,7 @@ jobs: # run: make -C build -j # - name: Unit tests # run: ctest --test-dir build -L TESTLABEL -j 4 -# - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 +# - uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0 # if: always() # with: # name: log-gcc-${{ env.CXX }} @@ -111,7 +111,7 @@ jobs: run: | . /opt/intel/oneapi/setvars.sh ctest --test-dir build --output-on-failure -L TESTLABEL -j 4 - - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 + - uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0 if: always() with: name: log-pvc-impi-icpx-${{ matrix.config }} @@ -152,7 +152,7 @@ jobs: # - name: MP unit tests # # run: srun -p cluster scripts/run_command_on_compute_node.sh build/Testing/mptests.outerr.txt ctest --test-dir build -R ^mp-tests-sycl -L MP -j 4 # run: ctest --test-dir build -R ^mp-tests-sycl -L MP -j 4 -# - uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 +# - uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0 # if: always() # with: # name: log-ishmem-impi-icpx-${{ matrix.config }} From 35180a6f1798c1a42cc9d2dcc7c5b03e27298ffd Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 20 Sep 2024 15:43:13 +0200 Subject: [PATCH 21/35] Bump actions/setup-python from 5.1.0 to 5.2.0 (#832) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.1.0 to 5.2.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/82c7e631bb3cdc910f68e0081d67478d79c6982d...f677139bbe7f9c59b41e40162b753c062f5d49a3) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/pr.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index f927e82972..33c08eb94d 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -24,7 +24,7 @@ jobs: - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 - name: Ubuntu dependencies run: scripts/install-doxygen.sh - - uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0 + - uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0 with: python-version: '3.10' cache: 'pip' @@ -187,7 +187,7 @@ jobs: if: ${{ github.ref == 'refs/heads/main' }} steps: - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 - - uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0 + - uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0 with: python-version: '3.10' cache: 'pip' From 57c9ac933a7f5693e16f88e78a02c40581c04060 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 20 Sep 2024 15:43:21 +0200 Subject: [PATCH 22/35] Bump certifi from 2024.7.4 to 2024.8.30 (#830) Bumps [certifi](https://github.com/certifi/python-certifi) from 2024.7.4 to 2024.8.30. - [Commits](https://github.com/certifi/python-certifi/compare/2024.07.04...2024.08.30) --- updated-dependencies: - dependency-name: certifi dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 3be22e3826..84067aaaef 100644 --- a/requirements.txt +++ b/requirements.txt @@ -9,7 +9,7 @@ beautifulsoup4==4.12.3 binaryornot==0.4.4 boolean.py==4.0 breathe==4.35.0 -certifi==2024.7.4 +certifi==2024.8.30 cfgv==3.4.0 chardet==5.2.0 charset-normalizer==3.3.2 From fc5e66fe3c666cda147c2d9d4af3ead56516e1a3 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 20 Sep 2024 15:43:27 +0200 Subject: [PATCH 23/35] Bump pyyaml from 6.0.1 to 6.0.2 (#829) Bumps [pyyaml](https://github.com/yaml/pyyaml) from 6.0.1 to 6.0.2. - [Release notes](https://github.com/yaml/pyyaml/releases) - [Changelog](https://github.com/yaml/pyyaml/blob/main/CHANGES) - [Commits](https://github.com/yaml/pyyaml/compare/6.0.1...6.0.2) --- updated-dependencies: - dependency-name: pyyaml dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 84067aaaef..50332e9985 100644 --- a/requirements.txt +++ b/requirements.txt @@ -30,7 +30,7 @@ pydata-sphinx-theme==0.15.4 pyenchant==3.2.2 Pygments==2.18.0 python-debian==0.1.49 -PyYAML==6.0.1 +PyYAML==6.0.2 requests==2.32.3 reuse==4.0.3 snowballstemmer==2.2.0 From 2d3ec4dbdfb1930cec3331a7714be456e576e668 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 20 Sep 2024 15:43:36 +0200 Subject: [PATCH 24/35] Bump soupsieve from 2.5 to 2.6 (#828) Bumps [soupsieve](https://github.com/facelessuser/soupsieve) from 2.5 to 2.6. - [Release notes](https://github.com/facelessuser/soupsieve/releases) - [Commits](https://github.com/facelessuser/soupsieve/compare/2.5...2.6) --- updated-dependencies: - dependency-name: soupsieve dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 50332e9985..31d3b9f1c5 100644 --- a/requirements.txt +++ b/requirements.txt @@ -34,7 +34,7 @@ PyYAML==6.0.2 requests==2.32.3 reuse==4.0.3 snowballstemmer==2.2.0 -soupsieve==2.5 +soupsieve==2.6 Sphinx==8.0.2 sphinx-book-theme==1.1.3 sphinxcontrib-applehelp==2.0.0 From 10b76cc109c1b099120e49035b9af84a775834be Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 20 Sep 2024 15:43:55 +0200 Subject: [PATCH 25/35] Bump license-expression from 30.2.0 to 30.3.1 (#827) Bumps [license-expression](https://github.com/aboutcode-org/license-expression) from 30.2.0 to 30.3.1. - [Release notes](https://github.com/aboutcode-org/license-expression/releases) - [Changelog](https://github.com/aboutcode-org/license-expression/blob/main/CHANGELOG.rst) - [Commits](https://github.com/aboutcode-org/license-expression/compare/v30.2.0...v30.3.1) --- updated-dependencies: - dependency-name: license-expression dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 31d3b9f1c5..7ca9c48409 100644 --- a/requirements.txt +++ b/requirements.txt @@ -20,7 +20,7 @@ identify==2.6.0 idna==3.10 imagesize==1.4.1 Jinja2==3.1.4 -license-expression==30.2.0 +license-expression==30.3.1 MarkupSafe==2.1.5 nodeenv==1.9.1 packaging==24.1 From dd3516be7be079171ea5598485665a4428e6bf20 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 25 Sep 2024 10:07:59 +0200 Subject: [PATCH 26/35] Bump virtualenv from 20.26.3 to 20.26.5 (#841) Bumps [virtualenv](https://github.com/pypa/virtualenv) from 20.26.3 to 20.26.5. - [Release notes](https://github.com/pypa/virtualenv/releases) - [Changelog](https://github.com/pypa/virtualenv/blob/main/docs/changelog.rst) - [Commits](https://github.com/pypa/virtualenv/compare/20.26.3...20.26.5) --- updated-dependencies: - dependency-name: virtualenv dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 7ca9c48409..be6b8a3808 100644 --- a/requirements.txt +++ b/requirements.txt @@ -46,4 +46,4 @@ sphinxcontrib-serializinghtml==2.0.0 sphinxcontrib-spelling==8.0.0 typing_extensions==4.12.2 urllib3==2.2.2 -virtualenv==20.26.3 +virtualenv==20.26.5 From 5555e6eb676245f45ffb16da20d8d1428fa2705a Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 25 Sep 2024 10:08:18 +0200 Subject: [PATCH 27/35] Bump filelock from 3.15.4 to 3.16.1 (#840) Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.15.4 to 3.16.1. - [Release notes](https://github.com/tox-dev/py-filelock/releases) - [Changelog](https://github.com/tox-dev/filelock/blob/main/docs/changelog.rst) - [Commits](https://github.com/tox-dev/py-filelock/compare/3.15.4...3.16.1) --- updated-dependencies: - dependency-name: filelock dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index be6b8a3808..2169b7bce7 100644 --- a/requirements.txt +++ b/requirements.txt @@ -15,7 +15,7 @@ chardet==5.2.0 charset-normalizer==3.3.2 distlib==0.3.8 docutils==0.21.2 -filelock==3.15.4 +filelock==3.16.1 identify==2.6.0 idna==3.10 imagesize==1.4.1 From a1b07ee26ec92abdd2ac03b70b8ace6ce02aa995 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 25 Sep 2024 10:09:19 +0200 Subject: [PATCH 28/35] Bump platformdirs from 4.2.2 to 4.3.6 (#837) Bumps [platformdirs](https://github.com/tox-dev/platformdirs) from 4.2.2 to 4.3.6. - [Release notes](https://github.com/tox-dev/platformdirs/releases) - [Changelog](https://github.com/tox-dev/platformdirs/blob/main/CHANGES.rst) - [Commits](https://github.com/tox-dev/platformdirs/compare/4.2.2...4.3.6) --- updated-dependencies: - dependency-name: platformdirs dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 2169b7bce7..e652eb5b4c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -24,7 +24,7 @@ license-expression==30.3.1 MarkupSafe==2.1.5 nodeenv==1.9.1 packaging==24.1 -platformdirs==4.2.2 +platformdirs==4.3.6 pre-commit==3.8.0 pydata-sphinx-theme==0.15.4 pyenchant==3.2.2 From a36d177bfbb2b0aa5547d1e20de189f77a3366fc Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 26 Sep 2024 12:11:09 +0200 Subject: [PATCH 29/35] Bump identify from 2.6.0 to 2.6.1 (#839) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [identify](https://github.com/pre-commit/identify) from 2.6.0 to 2.6.1. - [Commits](https://github.com/pre-commit/identify/compare/v2.6.0...v2.6.1) --- updated-dependencies: - dependency-name: identify dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Łukasz Ślusarczyk <112692748+lslusarczyk@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index e652eb5b4c..029488b53b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -16,7 +16,7 @@ charset-normalizer==3.3.2 distlib==0.3.8 docutils==0.21.2 filelock==3.16.1 -identify==2.6.0 +identify==2.6.1 idna==3.10 imagesize==1.4.1 Jinja2==3.1.4 From 104e374143a94941e04ec1ba7f3e1d4eaf9c589b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 26 Sep 2024 12:11:22 +0200 Subject: [PATCH 30/35] Bump urllib3 from 2.2.2 to 2.2.3 (#838) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](https://github.com/urllib3/urllib3/compare/2.2.2...2.2.3) --- updated-dependencies: - dependency-name: urllib3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Łukasz Ślusarczyk <112692748+lslusarczyk@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 029488b53b..2c9c4ceab1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -45,5 +45,5 @@ sphinxcontrib-qthelp==2.0.0 sphinxcontrib-serializinghtml==2.0.0 sphinxcontrib-spelling==8.0.0 typing_extensions==4.12.2 -urllib3==2.2.2 +urllib3==2.2.3 virtualenv==20.26.5 From 02bbffb64546748f1c974ee9c67efeac3bbb18e8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filip=20G=C5=82=C4=99bocki?= Date: Fri, 27 Sep 2024 10:35:21 +0200 Subject: [PATCH 31/35] added count to mp algorithms (#812) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * added count to mp algorithms --------- Co-authored-by: Filip Głębocki --- include/dr/mp.hpp | 1 + include/dr/mp/algorithms/count.hpp | 126 +++++++++++++++++++++++++++++ test/gtest/common/count.cpp | 68 ++++++++++++++++ test/gtest/mp/CMakeLists.txt | 3 +- 4 files changed, 197 insertions(+), 1 deletion(-) create mode 100644 include/dr/mp/algorithms/count.hpp create mode 100644 test/gtest/common/count.cpp diff --git a/include/dr/mp.hpp b/include/dr/mp.hpp index 5e9729b16a..f9598bbcd8 100644 --- a/include/dr/mp.hpp +++ b/include/dr/mp.hpp @@ -65,6 +65,7 @@ #include #include #include +#include #include #include #include diff --git a/include/dr/mp/algorithms/count.hpp b/include/dr/mp/algorithms/count.hpp new file mode 100644 index 0000000000..3616cded70 --- /dev/null +++ b/include/dr/mp/algorithms/count.hpp @@ -0,0 +1,126 @@ +// SPDX-FileCopyrightText: Intel Corporation +// +// SPDX-License-Identifier: BSD-3-Clause + +#pragma once + +namespace dr::mp::__detail { + +inline auto add_counts(rng::forward_range auto &&r) { + rng::range_difference_t zero{}; + + return std::accumulate(rng::begin(r), rng::end(r), zero); +} + +inline auto count_if_local(rng::forward_range auto &&r, auto &&pred) { + if (mp::use_sycl()) { + dr::drlog.debug(" with DPL\n"); +#ifdef SYCL_LANGUAGE_VERSION + return std::count_if(mp::dpl_policy(), + dr::__detail::direct_iterator(rng::begin(r)), + dr::__detail::direct_iterator(rng::end(r)), pred); +#else + assert(false); +#endif + } else { + dr::drlog.debug(" with CPU\n"); + return std::count_if(std::execution::par_unseq, + dr::__detail::direct_iterator(rng::begin(r)), + dr::__detail::direct_iterator(rng::end(r)), pred); + } +} + +template +auto count_if(std::size_t root, bool root_provided, DR &&dr, auto &&pred) { + using count_type = rng::range_difference_t; + auto comm = mp::default_comm(); + + if (rng::empty(dr)) { + return count_type{}; + } + + dr::drlog.debug("Parallel count\n"); + + // Count within the local segments + auto count = [=](auto &&r) { + assert(rng::size(r) > 0); + return count_if_local(r, pred); + }; + auto locals = rng::views::transform(local_segments(dr), count); + auto local = add_counts(locals); + + std::vector all(comm.size()); + if (root_provided) { + // Everyone gathers to root, only root adds up the counts + comm.gather(local, std::span{all}, root); + if (root == comm.rank()) { + return add_counts(all); + } else { + return count_type{}; + } + } else { + // Everyone gathers and everyone adds up the counts + comm.all_gather(local, all); + return add_counts(all); + } +} + +} // namespace dr::mp::__detail + +namespace dr::mp { + +class count_fn_ { +public: + template + auto operator()(std::size_t root, DR &&dr, const T &value) const { + auto pred = [=](auto &&v) { return v == value; }; + return __detail::count_if(root, true, dr, pred); + } + + template + auto operator()(DR &&dr, const T &value) const { + auto pred = [=](auto &&v) { return v == value; }; + return __detail::count_if(0, false, dr, pred); + } + + template + auto operator()(std::size_t root, DI first, DI last, const T &value) const { + auto pred = [=](auto &&v) { return v == value; }; + return __detail::count_if(root, true, rng::subrange(first, last), pred); + } + + template + auto operator()(DI first, DI last, const T &value) const { + auto pred = [=](auto &&v) { return v == value; }; + return __detail::count_if(0, false, rng::subrange(first, last), pred); + } +}; + +inline constexpr count_fn_ count; + +class count_if_fn_ { +public: + template + auto operator()(std::size_t root, DR &&dr, auto &&pred) const { + return __detail::count_if(root, true, dr, pred); + } + + template + auto operator()(DR &&dr, auto &&pred) const { + return __detail::count_if(0, false, dr, pred); + } + + template + auto operator()(std::size_t root, DI first, DI last, auto &&pred) const { + return __detail::count_if(root, true, rng::subrange(first, last), pred); + } + + template + auto operator()(DI first, DI last, auto &&pred) const { + return __detail::count_if(0, false, rng::subrange(first, last), pred); + } +}; + +inline constexpr count_if_fn_ count_if; + +}; // namespace dr::mp diff --git a/test/gtest/common/count.cpp b/test/gtest/common/count.cpp new file mode 100644 index 0000000000..f6442055fd --- /dev/null +++ b/test/gtest/common/count.cpp @@ -0,0 +1,68 @@ +// SPDX-FileCopyrightText: Intel Corporation +// +// SPDX-License-Identifier: BSD-3-Clause + +#include "xp-tests.hpp" + +// Fixture +template class Count : public testing::Test { +protected: +}; + +TYPED_TEST_SUITE(Count, AllTypes); + +TYPED_TEST(Count, EmptyIf) { + std::vector vec; + + Ops1 ops(0); + + auto pred = [=](auto &&v) { return true; }; + + EXPECT_EQ(xp::count_if(ops.dist_vec, pred), 0); + EXPECT_EQ(std::count_if(ops.vec.begin(), ops.vec.end(), pred), + xp::count_if(ops.dist_vec, pred)); +} + +TYPED_TEST(Count, BasicFirstElem) { + std::vector vec{1, 2, 3, 1, 1, 3, 4, 1, 5, 6, 7}; + + Ops1 ops(vec.size()); + ops.vec = vec; + xp::copy(ops.vec, ops.dist_vec.begin()); + + auto value = *ops.vec.begin(); + + EXPECT_EQ(xp::count(ops.dist_vec, value), 4); + EXPECT_EQ(std::count(ops.vec.begin(), ops.vec.end(), value), + xp::count(ops.dist_vec, value)); +} + +TYPED_TEST(Count, BasicFirstElemIf) { + std::vector vec{1, 2, 3, 1, 1, 3, 4, 1, 5, 6, 7}; + + Ops1 ops(vec.size()); + ops.vec = vec; + xp::copy(ops.vec, ops.dist_vec.begin()); + + auto value = *vec.begin(); + auto pred = [=](auto &&v) { return v == value; }; + + EXPECT_EQ(xp::count_if(ops.dist_vec, pred), 4); + EXPECT_EQ(std::count_if(ops.vec.begin(), ops.vec.end(), pred), + xp::count_if(ops.dist_vec, pred)); +} + +TYPED_TEST(Count, FirstElemsIf) { + std::vector vec(20); + std::iota(vec.begin(), vec.end(), 0); + + Ops1 ops(vec.size()); + ops.vec = vec; + xp::copy(ops.vec, ops.dist_vec.begin()); + + auto pred = [=](auto &&v) { return v < 5; }; + + EXPECT_EQ(xp::count_if(ops.dist_vec, pred), 5); + EXPECT_EQ(std::count_if(ops.vec.begin(), ops.vec.end(), pred), + xp::count_if(ops.dist_vec, pred)); +} diff --git a/test/gtest/mp/CMakeLists.txt b/test/gtest/mp/CMakeLists.txt index 8138b35fb0..32f26d120a 100644 --- a/test/gtest/mp/CMakeLists.txt +++ b/test/gtest/mp/CMakeLists.txt @@ -11,6 +11,7 @@ add_executable( mp-tests.cpp ../common/all.cpp ../common/copy.cpp + ../common/count.cpp ../common/counted.cpp ../common/distributed_vector.cpp ../common/drop.cpp @@ -57,7 +58,7 @@ add_executable( add_executable(mp-quick-test mp-tests.cpp - ../common/equal.cpp + ../common/count.cpp ) # cmake-format: on From 84e7bfe8073f98fd3c8a7ce5fbd0dc39d0ff335c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 11 Oct 2024 11:17:34 +0200 Subject: [PATCH 32/35] Bump virtualenv from 20.26.5 to 20.26.6 (#843) Bumps [virtualenv](https://github.com/pypa/virtualenv) from 20.26.5 to 20.26.6. - [Release notes](https://github.com/pypa/virtualenv/releases) - [Changelog](https://github.com/pypa/virtualenv/blob/main/docs/changelog.rst) - [Commits](https://github.com/pypa/virtualenv/compare/20.26.5...20.26.6) --- updated-dependencies: - dependency-name: virtualenv dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 2c9c4ceab1..41b39e96ac 100644 --- a/requirements.txt +++ b/requirements.txt @@ -46,4 +46,4 @@ sphinxcontrib-serializinghtml==2.0.0 sphinxcontrib-spelling==8.0.0 typing_extensions==4.12.2 urllib3==2.2.3 -virtualenv==20.26.5 +virtualenv==20.26.6 From 05016ac8f1f75aefb81cdf5b3d82f54ddbbb6a11 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 11 Oct 2024 11:18:04 +0200 Subject: [PATCH 33/35] Bump actions/checkout from 4.1.7 to 4.2.0 (#842) Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.7 to 4.2.0. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/692973e3d937129bcbf40652eb9f2f61becf3332...d632683dd7b4114ad314bca15554477dd762a938) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/pr.yml | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 33c08eb94d..8c7825b920 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -21,7 +21,7 @@ jobs: runs-on: intel-ubuntu-latest timeout-minutes: 10 steps: - - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 + - uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0 - name: Ubuntu dependencies run: scripts/install-doxygen.sh - uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0 @@ -48,7 +48,7 @@ jobs: # env: # CXX: ${{ matrix.cxx }} # steps: -# - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 +# - uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0 # - name: Generate # run: cmake -B build # - name: Build @@ -82,7 +82,7 @@ jobs: CXX: icpx FI_PROVIDER: tcp steps: - - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 + - uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0 - name: Set up Intel APT repository run: | # https://www.intel.com/content/www/us/en/docs/oneapi/installation-guide-linux/2024-0/apt.html @@ -142,7 +142,7 @@ jobs: # env: # CXX: icpx # steps: -# - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 +# - uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0 # - name: Generate # run: cmake -B build -DCMAKE_BUILD_TYPE=${{ matrix.config }} -DENABLE_ISHMEM=on -DENABLE_L0=on -DENABLE_OFI=on -DOFI_PROVIDER=psm3 -DONETILE_PVC=on # - name: Build ISHMEM @@ -171,7 +171,7 @@ jobs: env: CXX: g++-13 steps: - - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 + - uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0 - run: | echo "creating DEB package is going to be here" @@ -186,7 +186,7 @@ jobs: SPHINXOPTS: -q -W if: ${{ github.ref == 'refs/heads/main' }} steps: - - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 + - uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0 - uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0 with: python-version: '3.10' @@ -198,7 +198,7 @@ jobs: run: | make -C doc/spec html - name: Checkout gh-pages - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7 + uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0 with: ref: gh-pages path: gh-pages From 0adda364978144992e8e50b4c5aa5747225f69ef Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 11 Oct 2024 09:36:36 +0000 Subject: [PATCH 34/35] Bump pre-commit from 3.8.0 to 4.0.0 (#844) Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.8.0 to 4.0.0. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](https://github.com/pre-commit/pre-commit/compare/v3.8.0...v4.0.0) --- updated-dependencies: - dependency-name: pre-commit dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 41b39e96ac..adb09f3307 100644 --- a/requirements.txt +++ b/requirements.txt @@ -25,7 +25,7 @@ MarkupSafe==2.1.5 nodeenv==1.9.1 packaging==24.1 platformdirs==4.3.6 -pre-commit==3.8.0 +pre-commit==4.0.0 pydata-sphinx-theme==0.15.4 pyenchant==3.2.2 Pygments==2.18.0 From 01b3e25c252c9a11b032ce2e437ea21678ee7302 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 21 Oct 2024 14:17:43 +0000 Subject: [PATCH 35/35] Bump markupsafe from 2.1.5 to 3.0.2 (#854) Bumps [markupsafe](https://github.com/pallets/markupsafe) from 2.1.5 to 3.0.2. - [Release notes](https://github.com/pallets/markupsafe/releases) - [Changelog](https://github.com/pallets/markupsafe/blob/main/CHANGES.rst) - [Commits](https://github.com/pallets/markupsafe/compare/2.1.5...3.0.2) --- updated-dependencies: - dependency-name: markupsafe dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index adb09f3307..04ba531974 100644 --- a/requirements.txt +++ b/requirements.txt @@ -21,7 +21,7 @@ idna==3.10 imagesize==1.4.1 Jinja2==3.1.4 license-expression==30.3.1 -MarkupSafe==2.1.5 +MarkupSafe==3.0.2 nodeenv==1.9.1 packaging==24.1 platformdirs==4.3.6