Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enable Lifted Product Code tests and reintroduce piracy #371

Merged
merged 3 commits into from
Sep 27, 2024
Merged

enable Lifted Product Code tests and reintroduce piracy #371

merged 3 commits into from
Sep 27, 2024

Conversation

Krastanov
Copy link
Member

@royess , it seems the tests in your PR were never actually running. Part of them were missing because the dictionary of codes was not set properly (you entered the public function as a key instead of the private type). Part of them were not enabled because they were pasted outside of the testitem definition.

These two issues are now fixed, but that led to actually having errors in the PR, partially due to the fix to the piracy issue that the PR had introduced.

Could you look into this sometime soon? In particular, (1) given how you are using the lift function, maybe we can completely drop the repr::Function and just rely on Nemo/Hecke; (2) we need a fix for the adjoint piracy (potentially having this upstreamed).

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 27, 2024 •
edited
Loading

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 27 Sep 2024 - 16:00
  • Baseline: 27 Sep 2024 - 16:07
  • Package commits:
  • Target: 65f82d
  • Baseline: 57a006
  • Julia commits:
  • Target: 6f3fdf
  • Baseline: 6f3fdf
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.91 (5%) ✅ 0.98 (1%) ✅
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.01 (5%) 0.97 (1%) ✅
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.02 (5%) 1.02 (1%) ❌
["pauli", "mul", "20000000"] 0.94 (5%) ✅ 1.00 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 0.89 (5%) ✅ 1.00 (1%)
["stabilizer", "trace", "stabilizer"] 1.05 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "compactification"]
  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 24.04.1 LTS
 uname: Linux 6.8.0-1014-azure #16-Ubuntu SMP Thu Aug 15 20:04:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3244 MHz 1857 s 0 s 156 s 5649 s 0 s
 #2 3243 MHz 2511 s 0 s 196 s 4952 s 0 s
 #3 3242 MHz 1963 s 0 s 163 s 5554 s 0 s
 #4 3237 MHz 2253 s 0 s 201 s 5201 s 0 s
 Memory: 15.615280151367188 GB (13271.51171875 MB free)
 Uptime: 775.54 sec
 Load Avg: 1.03 1.04 0.72
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 24.04.1 LTS
 uname: Linux 6.8.0-1014-azure #16-Ubuntu SMP Thu Aug 15 20:04:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3246 MHz 2457 s 0 s 212 s 9033 s 0 s
 #2 3242 MHz 3552 s 0 s 264 s 7884 s 0 s
 #3 3246 MHz 2863 s 0 s 224 s 8636 s 0 s
 #4 3240 MHz 3690 s 0 s 256 s 7751 s 0 s
 Memory: 15.615280151367188 GB (12798.953125 MB free)
 Uptime: 1180.23 sec
 Load Avg: 1.06 1.04 0.85
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 27 Sep 2024 - 16:0
  • Package commit: 65f82d
  • Julia commit: 6f3fdf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "compactification", "compact"] 6.017 ms (5%)
["circuitsim", "compactification", "no_compact"] 6.292 ms (5%) 281.25 KiB (1%) 6000
["circuitsim", "mctrajectories", "q1001_r1"] 15.453 ms (5%) 500.78 KiB (1%) 18017
["circuitsim", "mctrajectories", "q101_r1"] 171.983 μs (5%) 50.53 KiB (1%) 1816
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 14.172 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.583 μs (5%) 256 bytes (1%) 6
["circuitsim", "mctrajectories_union", "q1001_r1"] 14.024 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_union", "q101_r1"] 117.171 μs (5%) 256 bytes (1%) 6
["circuitsim", "pftrajectories", "q1001_r1"] 57.769 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 162.065 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.061 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 5.790 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 46.468 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 148.600 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.047 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.724 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.668 μs (5%)
["circuitsim", "pftrajectories_union", "q1001_r1"] 23.213 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 125.977 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.005 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.394 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.392 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_dense500_stab"] 5.485 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_destab"] 963.602 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_stab"] 406.806 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot_on_dense500_destab"] 49.583 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_dense500_stab"] 23.364 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_destab"] 26.260 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_stab"] 12.303 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "dense500_on_dense500_destab"] 11.402 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_dense500_stab"] 5.482 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_destab"] 813.160 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_stab"] 406.845 μs (5%) 512 bytes (1%) 4
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.432 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 699.035 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.174 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 579.100 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.708 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.334 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.668 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.334 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.663 ms (5%) 1.37 MiB (1%) 22167
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 3.702 ms (5%) 2.72 MiB (1%) 40038
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.043 ms (5%) 2.77 MiB (1%) 40380
["ecc", "evaluate_decoder", "shor_pybp_comm"] 24.557 ms (5%) 3.94 MiB (1%) 116576
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 49.914 ms (5%) 7.80 MiB (1%) 228192
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 49.973 ms (5%) 7.86 MiB (1%) 228622
["ecc", "evaluate_decoder", "shor_table_comm"] 386.197 μs (5%) 212.08 KiB (1%) 4087
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.105 ms (5%) 286.31 KiB (1%) 3214
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.433 ms (5%) 349.67 KiB (1%) 3644
["ecc", "evaluate_decoder", "toric8_bp_comm"] 806.421 ms (5%) 7.143 ms 171.38 MiB (1%) 588267
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.703 s (5%) 345.12 MiB (1%) 1197752
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.677 s (5%) 22.202 ms 354.74 MiB (1%) 1239793
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 396.368 ms (5%) 7.23 MiB (1%) 129336
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 816.417 ms (5%) 15.53 MiB (1%) 275566
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 822.576 ms (5%) 17.27 MiB (1%) 291111
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.794 ms (5%) 1.84 MiB (1%) 16937
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 15.288 ms (5%) 4.69 MiB (1%) 50773
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 22.517 ms (5%) 6.43 MiB (1%) 66318
["ecc", "evaluate_decoder", "toric8_table_comm"] 5.160 ms (5%) 1.63 MiB (1%) 16847
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 15.451 ms (5%) 4.26 MiB (1%) 50588
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 22.683 ms (5%) 6.00 MiB (1%) 66133
["pauli", "mul", "100"] 14.506 ns (5%)
["pauli", "mul", "1000"] 18.158 ns (5%)
["pauli", "mul", "100000"] 720.282 ns (5%)
["pauli", "mul", "20000000"] 137.759 μs (5%)
["stabilizer", "canon", "cano500"] 2.718 ms (5%)
["stabilizer", "canon", "diag_cano500"] 558.891 μs (5%)
["stabilizer", "canon", "diag_gott500"] 8.543 ms (5%) 5.26 MiB (1%) 24516
["stabilizer", "canon", "diag_rref500"] 420.200 μs (5%)
["stabilizer", "canon", "gott500"] 10.625 ms (5%) 5.27 MiB (1%) 24526
["stabilizer", "canon", "md_cano500"] 1.361 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.360 ms (5%)
["stabilizer", "canon", "rref500"] 2.647 ms (5%)
["stabilizer", "project", "destabilizer"] 13.055 μs (5%) 288 bytes (1%) 3
["stabilizer", "project", "stabilizer"] 5.931 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.348 ms (5%) 23.97 MiB (1%) 31
["stabilizer", "tensor", "pow5_20"] 2.837 μs (5%) 6.48 KiB (1%) 28
["stabilizer", "trace", "destabilizer"] 26.239 μs (5%) 80 bytes (1%) 1
["stabilizer", "trace", "stabilizer"] 21.871 μs (5%) 112 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "compactification"]
  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 24.04.1 LTS
 uname: Linux 6.8.0-1014-azure #16-Ubuntu SMP Thu Aug 15 20:04:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3244 MHz 1857 s 0 s 156 s 5649 s 0 s
 #2 3243 MHz 2511 s 0 s 196 s 4952 s 0 s
 #3 3242 MHz 1963 s 0 s 163 s 5554 s 0 s
 #4 3237 MHz 2253 s 0 s 201 s 5201 s 0 s
 Memory: 15.615280151367188 GB (13271.51171875 MB free)
 Uptime: 775.54 sec
 Load Avg: 1.03 1.04 0.72
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 27 Sep 2024 - 16:7
  • Package commit: 57a006
  • Julia commit: 6f3fdf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "compactification", "compact"] 5.978 ms (5%)
["circuitsim", "compactification", "no_compact"] 6.241 ms (5%) 281.25 KiB (1%) 6000
["circuitsim", "mctrajectories", "q1001_r1"] 15.252 ms (5%) 500.78 KiB (1%) 18017
["circuitsim", "mctrajectories", "q101_r1"] 169.268 μs (5%) 50.53 KiB (1%) 1816
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 14.023 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.664 μs (5%) 256 bytes (1%) 6
["circuitsim", "mctrajectories_union", "q1001_r1"] 14.054 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_union", "q101_r1"] 117.280 μs (5%) 256 bytes (1%) 6
["circuitsim", "pftrajectories", "q1001_r1"] 60.383 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 164.420 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.050 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.031 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 46.338 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 147.397 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.033 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.747 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.639 μs (5%)
["circuitsim", "pftrajectories_union", "q1001_r1"] 23.524 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 125.286 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 999.539 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.424 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.397 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_dense500_stab"] 5.521 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_destab"] 963.642 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_stab"] 406.754 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot_on_dense500_destab"] 49.603 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_dense500_stab"] 23.174 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_destab"] 26.350 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_stab"] 12.364 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "dense500_on_dense500_destab"] 11.400 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_dense500_stab"] 5.522 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_destab"] 814.745 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_stab"] 406.885 μs (5%) 512 bytes (1%) 4
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.420 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 698.844 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.160 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 579.110 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.648 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.344 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.618 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.334 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.641 ms (5%) 1.37 MiB (1%) 22151
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 3.731 ms (5%) 2.72 MiB (1%) 40054
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.455 ms (5%) 2.82 MiB (1%) 41020
["ecc", "evaluate_decoder", "shor_pybp_comm"] 24.438 ms (5%) 3.94 MiB (1%) 116576
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 50.218 ms (5%) 7.80 MiB (1%) 228192
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 49.430 ms (5%) 7.86 MiB (1%) 228622
["ecc", "evaluate_decoder", "shor_table_comm"] 389.043 μs (5%) 212.08 KiB (1%) 4087
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.102 ms (5%) 286.31 KiB (1%) 3214
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.433 ms (5%) 349.67 KiB (1%) 3644
["ecc", "evaluate_decoder", "toric8_bp_comm"] 814.586 ms (5%) 10.437 ms 171.97 MiB (1%) 590231
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.680 s (5%) 355.12 MiB (1%) 1231364
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.641 s (5%) 18.151 ms 349.33 MiB (1%) 1221593
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 397.368 ms (5%) 7.23 MiB (1%) 129336
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 815.775 ms (5%) 15.53 MiB (1%) 275566
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 813.598 ms (5%) 17.27 MiB (1%) 291111
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.834 ms (5%) 1.84 MiB (1%) 16937
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 15.408 ms (5%) 4.69 MiB (1%) 50773
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 22.552 ms (5%) 6.43 MiB (1%) 66318
["ecc", "evaluate_decoder", "toric8_table_comm"] 5.172 ms (5%) 1.63 MiB (1%) 16847
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 15.530 ms (5%) 4.26 MiB (1%) 50588
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 22.768 ms (5%) 6.00 MiB (1%) 66133
["pauli", "mul", "100"] 14.195 ns (5%)
["pauli", "mul", "1000"] 18.450 ns (5%)
["pauli", "mul", "100000"] 713.397 ns (5%)
["pauli", "mul", "20000000"] 147.157 μs (5%)
["stabilizer", "canon", "cano500"] 2.714 ms (5%)
["stabilizer", "canon", "diag_cano500"] 559.162 μs (5%)
["stabilizer", "canon", "diag_gott500"] 8.573 ms (5%) 5.26 MiB (1%) 24516
["stabilizer", "canon", "diag_rref500"] 421.513 μs (5%)
["stabilizer", "canon", "gott500"] 10.608 ms (5%) 5.27 MiB (1%) 24526
["stabilizer", "canon", "md_cano500"] 1.347 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.344 ms (5%)
["stabilizer", "canon", "rref500"] 2.624 ms (5%)
["stabilizer", "project", "destabilizer"] 13.055 μs (5%) 288 bytes (1%) 3
["stabilizer", "project", "stabilizer"] 6.041 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.522 ms (5%) 23.97 MiB (1%) 31
["stabilizer", "tensor", "pow5_20"] 2.959 μs (5%) 6.48 KiB (1%) 28
["stabilizer", "trace", "destabilizer"] 26.449 μs (5%) 80 bytes (1%) 1
["stabilizer", "trace", "stabilizer"] 20.789 μs (5%) 112 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "compactification"]
  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 24.04.1 LTS
 uname: Linux 6.8.0-1014-azure #16-Ubuntu SMP Thu Aug 15 20:04:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3246 MHz 2457 s 0 s 212 s 9033 s 0 s
 #2 3242 MHz 3552 s 0 s 264 s 7884 s 0 s
 #3 3246 MHz 2863 s 0 s 224 s 8636 s 0 s
 #4 3240 MHz 3690 s 0 s 256 s 7751 s 0 s
 Memory: 15.615280151367188 GB (12798.953125 MB free)
 Uptime: 1180.23 sec
 Load Avg: 1.06 1.04 0.85
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.85
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves user_shstk clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft
Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 27 Sep 2024 - 17:19
  • Baseline: 27 Sep 2024 - 17:26
  • Package commits:
  • Target: 20979c
  • Baseline: 57a006
  • Julia commits:
  • Target: 6f3fdf
  • Baseline: 6f3fdf
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "pftrajectories", "q1001_r100"] 1.08 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 1.26 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 1.24 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.06 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.91 (5%) ✅ 0.98 (1%) ✅
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 0.98 (5%) 1.03 (1%) ❌
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.03 (5%) 1.05 (1%) ❌
["pauli", "mul", "20000000"] 0.87 (5%) ✅ 1.00 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 0.87 (5%) ✅ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "compactification"]
  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.5 LTS
 uname: Linux 6.8.0-1014-azure #16~22.04.1-Ubuntu SMP Thu Aug 15 21:31:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3252 MHz 2227 s 0 s 160 s 6179 s 0 s
 #2 3243 MHz 2560 s 0 s 163 s 5849 s 0 s
 #3 2596 MHz 1823 s 0 s 158 s 6580 s 0 s
 #4 2445 MHz 1899 s 0 s 190 s 6478 s 0 s
 Memory: 15.615272521972656 GB (13169.0703125 MB free)
 Uptime: 860.6 sec
 Load Avg: 1.0 1.02 0.7
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.5 LTS
 uname: Linux 6.8.0-1014-azure #16~22.04.1-Ubuntu SMP Thu Aug 15 21:31:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 2798 MHz 2977 s 0 s 220 s 9384 s 0 s
 #2 3200 MHz 3881 s 0 s 234 s 8474 s 0 s
 #3 3242 MHz 2491 s 0 s 221 s 9866 s 0 s
 #4 3243 MHz 3113 s 0 s 239 s 9232 s 0 s
 Memory: 15.615272521972656 GB (12623.72265625 MB free)
 Uptime: 1262.64 sec
 Load Avg: 1.04 1.02 0.83
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 27 Sep 2024 - 17:19
  • Package commit: 20979c
  • Julia commit: 6f3fdf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "compactification", "compact"] 5.996 ms (5%)
["circuitsim", "compactification", "no_compact"] 6.216 ms (5%) 281.25 KiB (1%) 6000
["circuitsim", "mctrajectories", "q1001_r1"] 15.135 ms (5%) 500.78 KiB (1%) 18017
["circuitsim", "mctrajectories", "q101_r1"] 166.901 μs (5%) 50.53 KiB (1%) 1816
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 14.000 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.401 μs (5%) 256 bytes (1%) 6
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.954 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_union", "q101_r1"] 116.598 μs (5%) 256 bytes (1%) 6
["circuitsim", "pftrajectories", "q1001_r1"] 60.103 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 173.103 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.050 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.031 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 58.670 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 151.823 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.041 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.696 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 5.820 μs (5%)
["circuitsim", "pftrajectories_union", "q1001_r1"] 23.514 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 125.344 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 998.905 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.424 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.111 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_dense500_stab"] 5.582 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_destab"] 966.785 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_stab"] 407.010 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot_on_dense500_destab"] 49.593 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_dense500_stab"] 23.072 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_destab"] 26.209 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_stab"] 12.003 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "dense500_on_dense500_destab"] 11.159 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_dense500_stab"] 5.605 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_destab"] 813.730 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_stab"] 407.010 μs (5%) 512 bytes (1%) 4
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.423 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 697.602 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.162 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 578.270 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.658 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.344 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.598 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.334 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.764 ms (5%) 1.38 MiB (1%) 22279
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 3.702 ms (5%) 2.71 MiB (1%) 39990
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.040 ms (5%) 2.78 MiB (1%) 40436
["ecc", "evaluate_decoder", "shor_pybp_comm"] 24.772 ms (5%) 3.94 MiB (1%) 116576
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 49.999 ms (5%) 7.80 MiB (1%) 228192
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 50.729 ms (5%) 7.86 MiB (1%) 228622
["ecc", "evaluate_decoder", "shor_table_comm"] 387.354 μs (5%) 212.08 KiB (1%) 4087
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.109 ms (5%) 286.31 KiB (1%) 3214
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.442 ms (5%) 349.67 KiB (1%) 3644
["ecc", "evaluate_decoder", "toric8_bp_comm"] 869.006 ms (5%) 10.587 ms 172.72 MiB (1%) 592747
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.716 s (5%) 355.88 MiB (1%) 1233928
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.681 s (5%) 22.742 ms 357.48 MiB (1%) 1249009
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 399.097 ms (5%) 7.23 MiB (1%) 129336
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 835.330 ms (5%) 15.53 MiB (1%) 275566
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 826.444 ms (5%) 17.27 MiB (1%) 291111
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.837 ms (5%) 1.84 MiB (1%) 16937
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 15.337 ms (5%) 4.69 MiB (1%) 50773
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 22.567 ms (5%) 6.43 MiB (1%) 66318
["ecc", "evaluate_decoder", "toric8_table_comm"] 5.187 ms (5%) 1.63 MiB (1%) 16847
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 15.547 ms (5%) 4.26 MiB (1%) 50588
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 22.656 ms (5%) 6.00 MiB (1%) 66133
["pauli", "mul", "100"] 14.194 ns (5%)
["pauli", "mul", "1000"] 17.917 ns (5%)
["pauli", "mul", "100000"] 716.262 ns (5%)
["pauli", "mul", "20000000"] 172.072 μs (5%)
["stabilizer", "canon", "cano500"] 2.869 ms (5%)
["stabilizer", "canon", "diag_cano500"] 558.820 μs (5%)
["stabilizer", "canon", "diag_gott500"] 8.715 ms (5%) 5.26 MiB (1%) 24516
["stabilizer", "canon", "diag_rref500"] 422.449 μs (5%)
["stabilizer", "canon", "gott500"] 10.861 ms (5%) 5.27 MiB (1%) 24526
["stabilizer", "canon", "md_cano500"] 1.402 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.370 ms (5%)
["stabilizer", "canon", "rref500"] 2.690 ms (5%)
["stabilizer", "project", "destabilizer"] 13.345 μs (5%) 288 bytes (1%) 3
["stabilizer", "project", "stabilizer"] 6.001 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.219 ms (5%) 23.97 MiB (1%) 31
["stabilizer", "tensor", "pow5_20"] 2.796 μs (5%) 6.48 KiB (1%) 28
["stabilizer", "trace", "destabilizer"] 27.131 μs (5%) 80 bytes (1%) 1
["stabilizer", "trace", "stabilizer"] 21.911 μs (5%) 112 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "compactification"]
  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.5 LTS
 uname: Linux 6.8.0-1014-azure #16~22.04.1-Ubuntu SMP Thu Aug 15 21:31:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3252 MHz 2227 s 0 s 160 s 6179 s 0 s
 #2 3243 MHz 2560 s 0 s 163 s 5849 s 0 s
 #3 2596 MHz 1823 s 0 s 158 s 6580 s 0 s
 #4 2445 MHz 1899 s 0 s 190 s 6478 s 0 s
 Memory: 15.615272521972656 GB (13169.0703125 MB free)
 Uptime: 860.6 sec
 Load Avg: 1.0 1.02 0.7
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 27 Sep 2024 - 17:26
  • Package commit: 57a006
  • Julia commit: 6f3fdf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "compactification", "compact"] 6.017 ms (5%)
["circuitsim", "compactification", "no_compact"] 6.252 ms (5%) 281.25 KiB (1%) 6000
["circuitsim", "mctrajectories", "q1001_r1"] 15.010 ms (5%) 500.78 KiB (1%) 18017
["circuitsim", "mctrajectories", "q101_r1"] 170.439 μs (5%) 50.53 KiB (1%) 1816
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 14.030 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 117.970 μs (5%) 256 bytes (1%) 6
["circuitsim", "mctrajectories_union", "q1001_r1"] 14.087 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_union", "q101_r1"] 116.638 μs (5%) 256 bytes (1%) 6
["circuitsim", "pftrajectories", "q1001_r1"] 57.767 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 160.700 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.043 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 5.780 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 46.597 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 147.656 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.037 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.717 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.688 μs (5%)
["circuitsim", "pftrajectories_union", "q1001_r1"] 23.213 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 124.803 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 998.974 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.394 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.159 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_dense500_stab"] 5.605 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_destab"] 966.925 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_stab"] 406.960 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot_on_dense500_destab"] 49.532 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_dense500_stab"] 23.234 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_destab"] 26.280 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_stab"] 11.912 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "dense500_on_dense500_destab"] 11.126 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_dense500_stab"] 5.578 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_destab"] 814.139 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_stab"] 406.979 μs (5%) 512 bytes (1%) 4
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.423 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 697.772 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.162 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 578.229 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.668 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.334 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.598 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.334 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.666 ms (5%) 1.37 MiB (1%) 22231
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 3.759 ms (5%) 2.72 MiB (1%) 40134
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.428 ms (5%) 2.84 MiB (1%) 41252
["ecc", "evaluate_decoder", "shor_pybp_comm"] 25.484 ms (5%) 3.94 MiB (1%) 116576
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 52.489 ms (5%) 7.80 MiB (1%) 228192
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 51.643 ms (5%) 7.86 MiB (1%) 228622
["ecc", "evaluate_decoder", "shor_table_comm"] 383.275 μs (5%) 212.08 KiB (1%) 4087
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.103 ms (5%) 286.31 KiB (1%) 3214
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.436 ms (5%) 349.67 KiB (1%) 3644
["ecc", "evaluate_decoder", "toric8_bp_comm"] 854.756 ms (5%) 10.093 ms 172.56 MiB (1%) 592235
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.758 s (5%) 134.800 ms 344.55 MiB (1%) 1195848
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.632 s (5%) 17.613 ms 339.01 MiB (1%) 1186929
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 399.350 ms (5%) 7.23 MiB (1%) 129336
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 811.938 ms (5%) 15.53 MiB (1%) 275566
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 833.386 ms (5%) 17.27 MiB (1%) 291111
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.857 ms (5%) 1.84 MiB (1%) 16937
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 15.377 ms (5%) 4.69 MiB (1%) 50773
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 22.597 ms (5%) 6.43 MiB (1%) 66318
["ecc", "evaluate_decoder", "toric8_table_comm"] 5.146 ms (5%) 1.63 MiB (1%) 16847
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 15.463 ms (5%) 4.26 MiB (1%) 50588
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 22.708 ms (5%) 6.00 MiB (1%) 66133
["pauli", "mul", "100"] 13.893 ns (5%)
["pauli", "mul", "1000"] 18.489 ns (5%)
["pauli", "mul", "100000"] 715.646 ns (5%)
["pauli", "mul", "20000000"] 197.329 μs (5%)
["stabilizer", "canon", "cano500"] 2.868 ms (5%)
["stabilizer", "canon", "diag_cano500"] 558.523 μs (5%)
["stabilizer", "canon", "diag_gott500"] 8.697 ms (5%) 5.26 MiB (1%) 24516
["stabilizer", "canon", "diag_rref500"] 421.036 μs (5%)
["stabilizer", "canon", "gott500"] 10.832 ms (5%) 5.27 MiB (1%) 24526
["stabilizer", "canon", "md_cano500"] 1.379 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.387 ms (5%)
["stabilizer", "canon", "rref500"] 2.687 ms (5%)
["stabilizer", "project", "destabilizer"] 13.045 μs (5%) 288 bytes (1%) 3
["stabilizer", "project", "stabilizer"] 5.991 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.400 ms (5%) 23.97 MiB (1%) 31
["stabilizer", "tensor", "pow5_20"] 2.884 μs (5%) 6.48 KiB (1%) 28
["stabilizer", "trace", "destabilizer"] 26.750 μs (5%) 80 bytes (1%) 1
["stabilizer", "trace", "stabilizer"] 21.750 μs (5%) 112 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "compactification"]
  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.5 LTS
 uname: Linux 6.8.0-1014-azure #16~22.04.1-Ubuntu SMP Thu Aug 15 21:31:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 2798 MHz 2977 s 0 s 220 s 9384 s 0 s
 #2 3200 MHz 3881 s 0 s 234 s 8474 s 0 s
 #3 3242 MHz 2491 s 0 s 221 s 9866 s 0 s
 #4 3243 MHz 3113 s 0 s 239 s 9232 s 0 s
 Memory: 15.615272521972656 GB (12623.72265625 MB free)
 Uptime: 1262.64 sec
 Load Avg: 1.04 1.02 0.83
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.84
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves user_shstk clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft
Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 27 Sep 2024 - 17:19
  • Baseline: 27 Sep 2024 - 17:26
  • Package commits:
  • Target: 20979c
  • Baseline: 57a006
  • Julia commits:
  • Target: 6f3fdf
  • Baseline: 6f3fdf
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "pftrajectories", "q1001_r100"] 1.08 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 1.26 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 1.24 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.06 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.91 (5%) ✅ 0.98 (1%) ✅
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 0.98 (5%) 1.03 (1%) ❌
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.03 (5%) 1.05 (1%) ❌
["pauli", "mul", "20000000"] 0.87 (5%) ✅ 1.00 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 0.87 (5%) ✅ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "compactification"]
  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.5 LTS
 uname: Linux 6.8.0-1014-azure #16~22.04.1-Ubuntu SMP Thu Aug 15 21:31:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3252 MHz 2227 s 0 s 160 s 6179 s 0 s
 #2 3243 MHz 2560 s 0 s 163 s 5849 s 0 s
 #3 2596 MHz 1823 s 0 s 158 s 6580 s 0 s
 #4 2445 MHz 1899 s 0 s 190 s 6478 s 0 s
 Memory: 15.615272521972656 GB (13169.0703125 MB free)
 Uptime: 860.6 sec
 Load Avg: 1.0 1.02 0.7
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.5 LTS
 uname: Linux 6.8.0-1014-azure #16~22.04.1-Ubuntu SMP Thu Aug 15 21:31:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 2798 MHz 2977 s 0 s 220 s 9384 s 0 s
 #2 3200 MHz 3881 s 0 s 234 s 8474 s 0 s
 #3 3242 MHz 2491 s 0 s 221 s 9866 s 0 s
 #4 3243 MHz 3113 s 0 s 239 s 9232 s 0 s
 Memory: 15.615272521972656 GB (12623.72265625 MB free)
 Uptime: 1262.64 sec
 Load Avg: 1.04 1.02 0.83
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 27 Sep 2024 - 17:19
  • Package commit: 20979c
  • Julia commit: 6f3fdf
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "compactification", "compact"] 5.996 ms (5%)
["circuitsim", "compactification", "no_compact"] 6.216 ms (5%) 281.25 KiB (1%) 6000
["circuitsim", "mctrajectories", "q1001_r1"] 15.135 ms (5%) 500.78 KiB (1%) 18017
["circuitsim", "mctrajectories", "q101_r1"] 166.901 μs (5%) 50.53 KiB (1%) 1816
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 14.000 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.401 μs (5%) 256 bytes (1%) 6
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.954 ms (5%) 512 bytes (1%) 7
["circuitsim", "mctrajectories_union", "q101_r1"] 116.598 μs (5%) 256 bytes (1%) 6
["circuitsim", "pftrajectories", "q1001_r1"] 60.103 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 173.103 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.050 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.031 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 58.670 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 151.823 μs (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.041 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.696 ms (5%)
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 5.820 μs (5%)
["circuitsim", "pftrajectories_union", "q1001_r1"] 23.514 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 125.344 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 998.905 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.424 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.111 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_dense500_stab"] 5.582 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_destab"] 966.785 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot250_on_diag500_stab"] 407.010 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "cnot_on_dense500_destab"] 49.593 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_dense500_stab"] 23.072 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_destab"] 26.209 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "cnot_on_diag500_stab"] 12.003 μs (5%) 368 bytes (1%) 5
["clifford", "dense", "dense500_on_dense500_destab"] 11.159 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_dense500_stab"] 5.605 ms (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_destab"] 813.730 μs (5%) 512 bytes (1%) 4
["clifford", "dense", "dense500_on_diag500_stab"] 407.010 μs (5%) 512 bytes (1%) 4
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.423 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 697.602 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.162 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 578.270 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.658 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.344 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.598 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.334 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.764 ms (5%) 1.38 MiB (1%) 22279
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 3.702 ms (5%) 2.71 MiB (1%) 39990
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.040 ms (5%) 2.78 MiB (1%) 40436
["ecc", "evaluate_decoder", "shor_pybp_comm"] 24.772 ms (5%) 3.94 MiB (1%) 116576
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 49.999 ms (5%) 7.80 MiB (1%) 228192
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 50.729 ms (5%) 7.86 MiB (1%) 228622
["ecc", "evaluate_decoder", "shor_table_comm"] 387.354 μs (5%) 212.08 KiB (1%) 4087
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.109 ms (5%) 286.31 KiB (1%) 3214
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.442 ms (5%) 349.67 KiB (1%) 3644
["ecc", "evaluate_decoder", "toric8_bp_comm"] 869.006 ms (5%) 10.587 ms 172.72 MiB (1%) 592747
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.716 s (5%) 355.88 MiB (1%) 1233928
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.681 s (5%) 22.742 ms 357.48 MiB (1%) 1249009
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 399.097 ms (5%) 7.23 MiB (1%) 129336
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 835.330 ms (5%) 15.53 MiB (1%) 275566
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 826.444 ms (5%) 17.27 MiB (1%) 291111
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.837 ms (5%) 1.84 MiB (1%) 16937
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 15.337 ms (5%) 4.69 MiB (1%) 50773
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 22.567 ms (5%) 6.43 MiB (1%) 66318
["ecc", "evaluate_decoder", "toric8_table_comm"] 5.187 ms (5%) 1.63 MiB (1%) 16847
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 15.547 ms (5%) 4.26 MiB (1%) 50588
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 22.656 ms (5%) 6.00 MiB (1%) 66133
["pauli", "mul", "100"] 14.194 ns (5%)
["pauli", "mul", "1000"] 17.917 ns (5%)
["pauli", "mul", "100000"] 716.262 ns (5%)
["pauli", "mul", "20000000"] 172.072 μs (5%)
["stabilizer", "canon", "cano500"] 2.869 ms (5%)
["stabilizer", "canon", "diag_cano500"] 558.820 μs (5%)
["stabilizer", "canon", "diag_gott500"] 8.715 ms (5%) 5.26 MiB (1%) 24516
["stabilizer", "canon", "diag_rref500"] 422.449 μs (5%)
["stabilizer", "canon", "gott500"] 10.861 ms (5%) 5.27 MiB (1%) 24526
["stabilizer", "canon", "md_cano500"] 1.402 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.370 ms (5%)
["stabilizer", "canon", "rref500"] 2.690 ms (5%)
["stabilizer", "project", "destabilizer"] 13.345 μs (5%) 288 bytes (1%) 3
["stabilizer", "project", "stabilizer"] 6.001 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.219 ms (5%) 23.97 MiB (1%) 31
["stabilizer", "tensor", "pow5_20"] 2.796 μs (5%) 6.48 KiB (1%) 28
["stabilizer", "trace", "destabilizer"] 27.131 μs (5%) 80 bytes (1%) 1
["stabilizer", "trace", "stabilizer"] 21.911 μs (5%) 112 bytes (1%) 2

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "compactification"]
  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.5 LTS
 uname: Linux 6.8.0-1014-azure #16~22.04.1-Ubuntu SMP Thu Aug 15 21:31:41 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3252 MHz 2227 s 0 s 160 s 6179 s 0 s
 #2 3243 MHz 2560 s 0 s 163 s 5849 s 0 s
 #3 2596 MHz 1823 s 0 s 158 s 6580 s 0 s
 #4 2445 MHz 1899 s 0 s 190 s 6478 s 0 s
 Memory: 15.615272521972656 GB (13169.0703125 MB free)
 Uptime: 860.6 sec
 Load Avg: 1.0 1.02 0.7
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmar...[Comment body truncated]

@royess
Copy link
Contributor

royess commented Sep 27, 2024

Thanks for letting me know this! I think these tests were missed at some late point when I made mistakes in a merge.

For (1), I want to first note that the lift that I defined and Nemo.lift are different and unrelated. My lift maps a matrix of Hecke group algebra elements to a binary matrix, while Nemo.lift works on Nemo elements for type conversion. I think my use of the same name causes this confusion, and we may want to change the name.

Whether to drop the repr::Function is a separate issue. It is now defaulted to be representation_matrix from Hecke. I introduced this freedom because the representation can be more efficient than the default one, as detailed in the docstring of the struct LiftedCode.

For (2), I think the problem is that the "adjoint" or "conjugate" of a group algebra element does not seem to be a standard math notation. But here, it would be convenient to define the "conjugate" of a group algebra element by inversing all of its group elements and keeping the coefficients. This convention is also seen in the literature on lifted product codes.

To me, it also looks like a naming problem. To resolve the piracy issue, we may just change the name "adjoint" and make it an internal function. Doing so would also require adjustment of hgp, i.e., the hypergraph product.

@Krastanov
Copy link
Member Author

Sounds good! I removed the lift piracy (what you did is frequently referred to as a "pun" in the julia ecosystem as you used a function that has a good name to implement a method that does not match how the function is already used).

Check the last commit I added -- it simplified the representation code a bit in order to make it possible to directly provide a Hecke function as default_repr. In terms of an API, I prefer to stick to this as anyone seriously working on these codes will be mostly using Hecke anyway and this library will be used only after the code has been converted to a tableau.

Thus, I think the only thing left to do is fix the adjoint piracy. I like the suggestion you made for a fix, but when I tried to implement it I got something wrong (I do not have a very good understanding of Nemo and Hecke). Please submit it when you have a chance (it is not urgent though, no need to rush)

@Krastanov
Copy link
Member Author

a side note -- I changed one of your nested vcat/hcat statements to hvcat(size(x,2), transpose(x)...) -- could you please double check I did not get this wrong? It is shorter and I think a bit neater, but I am not sure whether you meant the same operation.

@Krastanov Krastanov merged commit 4e06c01 into master Sep 27, 2024
15 of 16 checks passed
@Krastanov Krastanov deleted the lpcodefix branch September 27, 2024 17:40
@royess
Copy link
Contributor

royess commented Oct 1, 2024

@Krastanov I have just locally fixed the piracy issue and will soon make a PR for it with some documentation enhancement. (I was previously not aware of the "pun" problem. Thanks for letting me know!)

The hvcat is correctly used there.

@royess royess mentioned this pull request Oct 16, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Krastanov @royess