Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MOD-3535] snapshots capture references to modal objects which become #2164

Conversation

thundergolfer
Copy link
Contributor

@thundergolfer thundergolfer commented Aug 29, 2024

Describe your changes

  • MOD-3535
  • I've also added an integration test in the other repo (#15223).

In cases where a Modal object is looked up in global scope or in an @enter lifecycle function, memory snapshotting will freeze that object's ID and reuse it in perpetuity, not invalidating the reference if it creates a runtime error after restore.

f = modal.Function.lookup(...)
|
╰───> FunctionGet ──> self.object_id = fu-123  # can go stale!

This PR changes it so that upon restore these lookups must be redone. This is a small performance loss but maintains correctness.

Backward/forward compatibility checks

Check these boxes or delete any item (or this section) if not relevant for this PR.

  • Client+Server: this change is compatible with old servers
  • Client forward compatibility: this change ensures client can accept data intended for later versions of itself

Note on protobuf: protobuf message changes in one place may have impact to
multiple entities (client, server, worker, database). See points above.


@thundergolfer thundergolfer force-pushed the jonathon/mod-3535-snapshots-capture-references-to-modal-objects-which-become branch from 4388725 to 196dfb5 Compare August 30, 2024 00:38
Copy link
Contributor

@mwaskom mwaskom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Random thing this just made me wonder about — what happens if you snapshot/restore across client version boundaries?

modal/object.py Outdated Show resolved Hide resolved
modal/client.py Outdated Show resolved Hide resolved
@thundergolfer
Copy link
Contributor Author

thundergolfer commented Sep 3, 2024

what happens if you snapshot/restore across client version boundaries?

@mwaskom the client version will be fixed across snapshot/restore because it's the same container with the same client mount. If the Function's client mount changes the the snapshot would be invalidated.

modal/object.py Outdated Show resolved Hide resolved
modal/object.py Show resolved Hide resolved
@thundergolfer thundergolfer merged commit 02e1590 into main Sep 3, 2024
22 checks passed
@thundergolfer thundergolfer deleted the jonathon/mod-3535-snapshots-capture-references-to-modal-objects-which-become branch September 3, 2024 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants