fix: deferred checkpoints optimization #1973

ctian1 · 2025-01-23T01:53:50Z

Currently, the entire ExecutionState.proof_stream is being cloned for every checkpoint, which is very wasteful for programs with many deferred proofs. Since it's only used as a runtime check, we can skip the clone and verification with checkpoints.

ratankaliani · 2025-01-23T03:26:55Z

.github/workflows/release.yml

@@ -33,7 +33,7 @@ jobs:
          pull_token: ${{ secrets.PULL_TOKEN }}

      # If it's a nightly release, tag with the release time. If the tag is `main`, we want to use
-      # `latest` as the tag name. Else, use the tag name as is.
+      # `latest` as the tag name. else, use the tag name as is.


nit: keep as is

ratankaliani · 2025-01-23T03:27:17Z

.github/workflows/release.yml

Are these changes relevant?

ratankaliani · 2025-01-23T03:27:28Z

.github/workflows/toolchain-ec2.yml

Are these changes relevant?

ratankaliani

Remove the release workflows, otherwise LGTM.

Co-authored-by: N <[email protected]>

* chore: bump docker image * chore: bump patches * fix: workflow docker gnark install toolchain * fix: bump examples patch * fix: clean up install logic * fix: remove ed-consesnus patch * fix: test mod vis * fix: examples * fix: docker ci (#230) * chore: bump docker image * chore: bump patches * fix: workflow docker gnark install toolchain * fix: bump examples patch * fix: clean up install logic * fix * fix ci --------- Co-authored-by: nhtyy <[email protected]> * doc updates * chore: book + tracing (#1932) * fix: book deploy workflow * add versioned docs * chore: don't skip deploy step * try symlink gcc per rocksdb rec * try add fix for ARM musl targets * fix: need seperate musl setups arm vs x86 * combine old arm musl setup * feat: v4.0.1 (#1941) Co-authored-by: Ratan Kaliani <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix: macos-12 runners deprecated * improve docs regarding new memory argument * resolve comments * fix(book): commit yarn lock file (#1960) * feat: display the correct proof type in SP1ProofWithPublicValues::bytes() (#1964) * fix(ci): cache paths not found in book workflow * fix(ci): correct cache usage for book (#1965) * fix(ci): fix big int version to avoid msrv (#1966) * chore: include rustc --version stderr in panic (#1968) * fix(sdk): implemented verify for EnvProver (#1963) * perf: rm vec allocs from words conversions (#1972) * feat(sdk): proof timeout fix (#1971) * feat: don't checkout submodules when doing `cargo prove new --evm` (#1976) * fix(sdk): retry network request proof (#1977) * feat(sdk): Turn off `alloy-*` crate features by default (#1980) * docs: add audit report for v4, fix typo (#1988) * docs: improve precompile security docs (#1991) * fix: Write diagnostics to stderr (vs stdout) (#1978) * fix(examples): check next validators hash in tendermint program (#1992) * feat(docker-build): pass additional workspace directory (#1975) * chore: ignore generated by Intellij-based IDEs. (#1986) * docs: add GitHub token usage example in installation guide (#1984) * feat: remove deprecated #[clap] attribute (#1898) * chore: rm redundant to_vec (#1970) * chore(clippy): misc clippy fixes (#1969) * fix: use downloadable go directive 1.22.0 (#1967) * chore: reorder impl members to match trait definition (#1985) * fix: deferred checkpoints optimization (#1973) Co-authored-by: N <[email protected]> * fix(ci): clippy + fmt (#1997) * fix: typo in prover network docs (#1998) * test: check syscalls are actually emitted (#1981) * perf(prover): powers of alpha (#2000) Co-authored-by: nhtyy <[email protected]> Co-authored-by: Yuwen Zhang <[email protected]> Co-authored-by: jtguibas <[email protected]> Co-authored-by: Ratan Kaliani <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: rkm0959 <[email protected]> Co-authored-by: Aurélien <[email protected]> Co-authored-by: Matthias Seitz <[email protected]> Co-authored-by: srdtrk <[email protected]> Co-authored-by: Matt Stam <[email protected]> Co-authored-by: Ivan Mikushin <[email protected]> Co-authored-by: Abishek Bashyal <[email protected]> Co-authored-by: DevOrbitlabs <[email protected]> Co-authored-by: Gengar <[email protected]> Co-authored-by: crStiv <[email protected]> Co-authored-by: Chris T. <[email protected]> * nit --------- Co-authored-by: nhtyy <[email protected]> Co-authored-by: Yuwen Zhang <[email protected]> Co-authored-by: jtguibas <[email protected]> Co-authored-by: Ratan Kaliani <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: rkm0959 <[email protected]> Co-authored-by: Aurélien <[email protected]> Co-authored-by: Matthias Seitz <[email protected]> Co-authored-by: srdtrk <[email protected]> Co-authored-by: Matt Stam <[email protected]> Co-authored-by: Ivan Mikushin <[email protected]> Co-authored-by: Abishek Bashyal <[email protected]> Co-authored-by: DevOrbitlabs <[email protected]> Co-authored-by: Gengar <[email protected]> Co-authored-by: crStiv <[email protected]> Co-authored-by: Chris T. <[email protected]>

* chore: bump docker image * chore: bump patches * fix: workflow docker gnark install toolchain * fix: bump examples patch * fix: clean up install logic * fix: remove ed-consesnus patch * fix: test mod vis * fix: examples * fix: docker ci (#230) * chore: bump docker image * chore: bump patches * fix: workflow docker gnark install toolchain * fix: bump examples patch * fix: clean up install logic * fix * fix ci --------- Co-authored-by: nhtyy <[email protected]> * doc updates * chore: book + tracing (#1932) * fix: book deploy workflow * add versioned docs * chore: don't skip deploy step * try symlink gcc per rocksdb rec * try add fix for ARM musl targets * fix: need seperate musl setups arm vs x86 * combine old arm musl setup * feat: v4.0.1 (#1941) Co-authored-by: Ratan Kaliani <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix: macos-12 runners deprecated * improve docs regarding new memory argument * resolve comments * fix(book): commit yarn lock file (#1960) * feat: display the correct proof type in SP1ProofWithPublicValues::bytes() (#1964) * fix(ci): cache paths not found in book workflow * fix(ci): correct cache usage for book (#1965) * fix(ci): fix big int version to avoid msrv (#1966) * chore: include rustc --version stderr in panic (#1968) * fix(sdk): implemented verify for EnvProver (#1963) * perf: rm vec allocs from words conversions (#1972) * feat(sdk): proof timeout fix (#1971) * feat: don't checkout submodules when doing `cargo prove new --evm` (#1976) * fix(sdk): retry network request proof (#1977) * feat(sdk): Turn off `alloy-*` crate features by default (#1980) * docs: add audit report for v4, fix typo (#1988) * docs: improve precompile security docs (#1991) * fix: Write diagnostics to stderr (vs stdout) (#1978) * fix(examples): check next validators hash in tendermint program (#1992) * feat(docker-build): pass additional workspace directory (#1975) * chore: ignore generated by Intellij-based IDEs. (#1986) * docs: add GitHub token usage example in installation guide (#1984) * feat: remove deprecated #[clap] attribute (#1898) * chore: rm redundant to_vec (#1970) * chore(clippy): misc clippy fixes (#1969) * fix: use downloadable go directive 1.22.0 (#1967) * chore: reorder impl members to match trait definition (#1985) * fix: deferred checkpoints optimization (#1973) Co-authored-by: N <[email protected]> * fix(ci): clippy + fmt (#1997) * fix: typo in prover network docs (#1998) * test: check syscalls are actually emitted (#1981) * perf(prover): powers of alpha (#2000) Co-authored-by: nhtyy <[email protected]> Co-authored-by: Yuwen Zhang <[email protected]> Co-authored-by: jtguibas <[email protected]> Co-authored-by: Ratan Kaliani <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: rkm0959 <[email protected]> Co-authored-by: Aurélien <[email protected]> Co-authored-by: Matthias Seitz <[email protected]> Co-authored-by: srdtrk <[email protected]> Co-authored-by: Matt Stam <[email protected]> Co-authored-by: Ivan Mikushin <[email protected]> Co-authored-by: Abishek Bashyal <[email protected]> Co-authored-by: DevOrbitlabs <[email protected]> Co-authored-by: Gengar <[email protected]> Co-authored-by: crStiv <[email protected]> Co-authored-by: Chris T. <[email protected]> * fix: counts and small shape fixing (#2007) * patch lock --------- Co-authored-by: nhtyy <[email protected]> Co-authored-by: Yuwen Zhang <[email protected]> Co-authored-by: jtguibas <[email protected]> Co-authored-by: Ratan Kaliani <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: rkm0959 <[email protected]> Co-authored-by: Aurélien <[email protected]> Co-authored-by: Matthias Seitz <[email protected]> Co-authored-by: srdtrk <[email protected]> Co-authored-by: Matt Stam <[email protected]> Co-authored-by: Ivan Mikushin <[email protected]> Co-authored-by: Abishek Bashyal <[email protected]> Co-authored-by: DevOrbitlabs <[email protected]> Co-authored-by: Gengar <[email protected]> Co-authored-by: crStiv <[email protected]> Co-authored-by: Chris T. <[email protected]> Co-authored-by: umadayal <[email protected]>

nhtyy and others added 4 commits January 17, 2025 18:09

fix: macos-12 runners deprecated (#1954)

f83e147

fix(ci): main release ci should sp1up from latest tag

c4db7f3

fix(ci): ec2 toolchain tests should use latest tag for main

9d8e067

opt: deferred proofs checkpoint perf

b28cdf4

ctian1 changed the title ~~opt: deferred checkpoints~~ fix: deferred checkpoints optimization Jan 23, 2025

ratankaliani reviewed Jan 23, 2025

View reviewed changes

.github/workflows/release.yml

Copy link

Member

ratankaliani Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these changes relevant?

ratankaliani reviewed Jan 23, 2025

View reviewed changes

.github/workflows/toolchain-ec2.yml

Copy link

Member

ratankaliani Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these changes relevant?

ratankaliani approved these changes Jan 23, 2025

View reviewed changes

leruaa merged commit 73d08ca into dev Jan 28, 2025
14 of 15 checks passed

leruaa deleted the chris/deferred-checkpoint-opts branch January 28, 2025 16:53

umadayal pushed a commit that referenced this pull request Jan 29, 2025

fix: deferred checkpoints optimization (#1973)

1dda2c0

Co-authored-by: N <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: deferred checkpoints optimization #1973

fix: deferred checkpoints optimization #1973

ctian1 commented Jan 23, 2025 •

edited

Loading

ratankaliani Jan 23, 2025

ratankaliani Jan 23, 2025

ratankaliani Jan 23, 2025

ratankaliani left a comment

fix: deferred checkpoints optimization #1973

fix: deferred checkpoints optimization #1973

Conversation

ctian1 commented Jan 23, 2025 • edited Loading

ratankaliani Jan 23, 2025

Choose a reason for hiding this comment

ratankaliani Jan 23, 2025

Choose a reason for hiding this comment

ratankaliani Jan 23, 2025

Choose a reason for hiding this comment

ratankaliani left a comment

Choose a reason for hiding this comment

ctian1 commented Jan 23, 2025 •

edited

Loading