Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support global storage as cache locations #351

Closed
adammoody opened this issue Jun 4, 2021 · 2 comments
Closed

support global storage as cache locations #351

adammoody opened this issue Jun 4, 2021 · 2 comments

Comments

@adammoody
Copy link
Contributor

While it should be possible to use global storage as a cache level, we currently have some hard-coded assumptions and scalability issues because SCR generally assumes cache locations are using node-local storage.

For example, when inspecting the cache on restart, one rank is responsible for distributing work. With global storage, this means rank 0 will have to talk with every process in the job, which becomes an N-to-1 bottleneck.

We should extend SCR to efficiently support global cache locations.

Related to #133

@adammoody
Copy link
Contributor Author

Replaced with more specific issues:
#489
#493
#494

@mShining
Copy link

mShining commented Apr 21, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants