diff --git a/.circleci/config.yml b/.circleci/config.yml index 21a25993..c48679e1 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -211,7 +211,7 @@ workflows: - deploy-tag: filters: tags: - only: /^\d+\.\d+\.\d+$/ # only release tags, e.g 2.7.0 + only: /^\d+\.\d+\.\d+$/ # only release tags, e.g 3.0.0 # See https://discuss.circleci.com/t/tag-not-triggered-by-circleci-not-filter-in-workflow/32036 branches: ignore: /.*/ # Ignore all branches, otherwise the job is triggered by any git push diff --git a/Changelog.md b/Changelog.md index 5108eb82..707080cd 100644 --- a/Changelog.md +++ b/Changelog.md @@ -1,3 +1,36 @@ +# Version 3.0.0 + +### Optimizations: + +Implemented new distributed matrix multiplication algorithm for calculating Q matrix. +It employs MPI shared windows, Chinese Remainder Theorem, [FLINT](https://flintlib.org/) and BLAS libraries. + +Our benchmarks for large SDPB problems demonstrated more than **~2.5x** overall program speedup and much better performance scaling with increasing number of CPUs and nodes, as compared to the 2.7.0 release. +In addition, improved RAM usage in the new algorithm now allows to solve even larger problems where Q matrix does not fit into single node memory. + +See [#142](https://github.com/davidsd/sdpb/pull/142), [#212](https://github.com/davidsd/sdpb/pull/212). + +### New features: + +- New option `--maxSharedMemory` allowing to reduce memory usage in the new matrix multiplication algorithm. If the limit is not set, it is calculated automatically. See [#209](https://github.com/davidsd/sdpb/pull/209), [#229](https://github.com/davidsd/sdpb/pull/229). +- Added new `--verbosity trace` level. See [#230](https://github.com/davidsd/sdpb/pull/230). + +### Other improvements: + +- Print iterations data and condition numbers to `iterations.json` in the output folder. See [#228](https://github.com/davidsd/sdpb/pull/228), [#231](https://github.com/davidsd/sdpb/pull/231), [#232](https://github.com/davidsd/sdpb/pull/232). +- In debug mode, write profiling data for both timing run and actual run, see [#215](https://github.com/davidsd/sdpb/pull/215). +- In debug mode, print maximal memory usage [#231](https://github.com/davidsd/sdpb/pull/231). +- Graceful exit on SIGTERM, see [#208](https://github.com/davidsd/sdpb/pull/208). +- Build multiplatform Docker image (AMD64+ARM64) on CircleCI, see [#225](https://github.com/davidsd/sdpb/pull/225). + + +### New dependencies: + +- [FLINT](https://flintlib.org/) library +- CBLAS implementation, e.g. [OpenBLAS](https://www.openblas.net/) + +See https://github.com/davidsd/sdpb/releases/tag/3.0.0 for the full changelog. + # Version 2.7.0 ### Breaking changes: diff --git a/docs/Docker.md b/docs/Docker.md index 5af0c083..280206b7 100644 --- a/docs/Docker.md +++ b/docs/Docker.md @@ -20,9 +20,9 @@ can be downloaded from [Docker Hub](https://hub.docker.com/r/bootstrapcollaborat docker pull bootstrapcollaboration/sdpb:master -You can also download image for a specific release, e.g., 2.7.0: +You can also download image for a specific release, e.g., 3.0.0: - docker pull bootstrapcollaboration/sdpb:2.7.0 + docker pull bootstrapcollaboration/sdpb:3.0.0 The list of all available tags can be found in [bootstrapcollaboration/sdpb](https://hub.docker.com/r/bootstrapcollaboration/sdpb/tags) repo on Docker Hub. diff --git a/docs/SDPB_Manual/SDPB-Manual.pdf b/docs/SDPB_Manual/SDPB-Manual.pdf index 5bed2e21..0a0dec64 100644 Binary files a/docs/SDPB_Manual/SDPB-Manual.pdf and b/docs/SDPB_Manual/SDPB-Manual.pdf differ diff --git a/docs/SDPB_Manual/SDPB-Manual.tex b/docs/SDPB_Manual/SDPB-Manual.tex index 77f74c66..a1f614f6 100644 --- a/docs/SDPB_Manual/SDPB-Manual.tex +++ b/docs/SDPB_Manual/SDPB-Manual.tex @@ -38,7 +38,7 @@ {\Large \begin{center} -{\bf SDPB 2.7.0 \\\vspace{.1in}} +{\bf SDPB 3.0.0 \\\vspace{.1in}} \end{center} } \begin{center} diff --git a/docs/Singularity.md b/docs/Singularity.md index 6dbeecf1..a81476e5 100644 --- a/docs/Singularity.md +++ b/docs/Singularity.md @@ -54,21 +54,21 @@ and push it to [local registry](https://docs.docker.com/registry/): ## Run Singularity image Singularity should automatically mount your home directory. If your -Singularity image is named `sdpb-2.7.0.sif`, you can invoke the SDPB +Singularity image is named `sdpb-3.0.0.sif`, you can invoke the SDPB programs by prepending the command with - singularity exec sdpb-2.7.0.sif + singularity exec sdpb-3.0.0.sif So to convert the JSON input file at `/home/user/input.json`, run the command - singularity exec sdpb-2.7.0.sif mpirun -n 4 pmp2sdp --precision 1024 -i /home/user/input.json -o /home/user/input + singularity exec sdpb-3.0.0.sif mpirun -n 4 pmp2sdp --precision 1024 -i /home/user/input.json -o /home/user/input This uses 4 cores when running pmp2sdp. You can change that number to match your own machine. To find a primal-dual solution, - singularity exec sdpb-2.7.0.sif mpirun -n 4 sdpb --precision=1024 -s /home/user/input + singularity exec sdpb-3.0.0.sif mpirun -n 4 sdpb --precision=1024 -s /home/user/input In theory, Singularity can be used to run jobs across multiple nodes. We have not been able to make that work yet. So for large, multi-node diff --git a/docs/site_installs/Boston.md b/docs/site_installs/Boston.md index 1872df45..1cd1b52b 100644 --- a/docs/site_installs/Boston.md +++ b/docs/site_installs/Boston.md @@ -32,12 +32,12 @@ You may list all available versions via Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), -and `sdpb-2.7.0` is a stable [2.7.0](https://github.com/davidsd/sdpb/releases/tag/2.7.0) release. +and `sdpb-3.0.0` is a stable [3.0.0](https://github.com/davidsd/sdpb/releases/tag/3.0.0) release. Examples below are for `sdpb-master`. -You may replace it with another version, e.g. `sdpb-2.7.0`. +You may replace it with another version, e.g. `sdpb-3.0.0`. In that case, please refer -to [2.7.0 documentation](https://github.com/davidsd/sdpb/blob/2.7.0/docs/site_installs/Boston.md). +to [3.0.0 documentation](https://github.com/davidsd/sdpb/blob/3.0.0/docs/site_installs/Boston.md). ## Run SDPB diff --git a/docs/site_installs/Caltech.md b/docs/site_installs/Caltech.md index 952e2044..09cd13a8 100644 --- a/docs/site_installs/Caltech.md +++ b/docs/site_installs/Caltech.md @@ -33,12 +33,12 @@ You may list all available versions via Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), -and `sdpb-2.7.0` is a stable [2.7.0](https://github.com/davidsd/sdpb/releases/tag/2.7.0) release. +and `sdpb-3.0.0` is a stable [3.0.0](https://github.com/davidsd/sdpb/releases/tag/3.0.0) release. Examples below are for `sdpb-master`. -You may replace it with another version, e.g. `sdpb-2.7.0`. +You may replace it with another version, e.g. `sdpb-3.0.0`. In that case, please refer -to [2.7.0 documentation](https://github.com/davidsd/sdpb/blob/2.7.0/docs/site_installs/Caltech.md). +to [3.0.0 documentation](https://github.com/davidsd/sdpb/blob/3.0.0/docs/site_installs/Caltech.md). ## Run SDPB diff --git a/docs/site_installs/Expanse.md b/docs/site_installs/Expanse.md index 665f3ab9..caf25e6b 100644 --- a/docs/site_installs/Expanse.md +++ b/docs/site_installs/Expanse.md @@ -32,12 +32,12 @@ You may list all available versions via Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), -and `sdpb-2.7.0` is a stable [2.7.0](https://github.com/davidsd/sdpb/releases/tag/2.7.0) release. +and `sdpb-3.0.0` is a stable [3.0.0](https://github.com/davidsd/sdpb/releases/tag/3.0.0) release. Examples below are for `sdpb-master`. -You may replace it with another version, e.g. `sdpb-2.7.0`. +You may replace it with another version, e.g. `sdpb-3.0.0`. In that case, please refer -to [2.7.0 documentation](https://github.com/davidsd/sdpb/blob/2.7.0/docs/site_installs/Expanse.md). +to [3.0.0 documentation](https://github.com/davidsd/sdpb/blob/3.0.0/docs/site_installs/Expanse.md). ## Run SDPB diff --git a/docs/site_installs/Harvard.md b/docs/site_installs/Harvard.md index 80cea754..3c1cb963 100644 --- a/docs/site_installs/Harvard.md +++ b/docs/site_installs/Harvard.md @@ -32,12 +32,12 @@ You may list all available versions via Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), -and `sdpb-2.7.0` is a stable [2.7.0](https://github.com/davidsd/sdpb/releases/tag/2.7.0) release. +and `sdpb-3.0.0` is a stable [3.0.0](https://github.com/davidsd/sdpb/releases/tag/3.0.0) release. Examples below are for `sdpb-master`. -You may replace it with another version, e.g. `sdpb-2.7.0`. +You may replace it with another version, e.g. `sdpb-3.0.0`. In that case, please refer -to [2.7.0 documentation](https://github.com/davidsd/sdpb/blob/2.7.0/docs/site_installs/Harvard.md). +to [3.0.0 documentation](https://github.com/davidsd/sdpb/blob/3.0.0/docs/site_installs/Harvard.md). ## Run SDPB diff --git a/docs/site_installs/Imperial.md b/docs/site_installs/Imperial.md index 5770386f..6e5abce5 100644 --- a/docs/site_installs/Imperial.md +++ b/docs/site_installs/Imperial.md @@ -33,12 +33,12 @@ You may list all available versions via Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), -and `sdpb-2.7.0` is a stable [2.7.0](https://github.com/davidsd/sdpb/releases/tag/2.7.0) release. +and `sdpb-3.0.0` is a stable [3.0.0](https://github.com/davidsd/sdpb/releases/tag/3.0.0) release. Examples below are for `sdpb-master`. -You may replace it with another version, e.g. `sdpb-2.7.0`. +You may replace it with another version, e.g. `sdpb-3.0.0`. In that case, please refer -to [2.7.0 documentation](https://github.com/davidsd/sdpb/blob/2.7.0/docs/site_installs/Imperial.md). +to [3.0.0 documentation](https://github.com/davidsd/sdpb/blob/3.0.0/docs/site_installs/Imperial.md). ## Run SDPB diff --git a/src/sdp_solve/SDP_Solver/run/bigint_syrk/Readme.md b/src/sdp_solve/SDP_Solver/run/bigint_syrk/Readme.md index b933d726..97ffa78a 100644 --- a/src/sdp_solve/SDP_Solver/run/bigint_syrk/Readme.md +++ b/src/sdp_solve/SDP_Solver/run/bigint_syrk/Readme.md @@ -77,7 +77,7 @@ In turn, Q is `DistMatrix` distributed over all cores. In the old SDPB algorithm, each core (or group of cores) was calculating a contribution to Q ( called `Q_group`) from its blocks. Adding up all Q_groups, one got the resulting Q -(see [synchronize_Q.cxx](https://github.com/davidsd/sdpb/blob/2.7.0/src/sdp_solve/SDP_Solver/run/step/initialize_schur_complement_solver/synchronize_Q.cxx)). +(see [synchronize_Q.cxx](https://github.com/davidsd/sdpb/blob/3.0.0/src/sdp_solve/SDP_Solver/run/step/initialize_schur_complement_solver/synchronize_Q.cxx)). This is inefficient in terms of memory, since we allocate memory for `Q_group` for each group of cores.