Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for existence of output file and short circuit unless force=True is passed to task. #10

Open
drewoldag opened this issue Jul 15, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@drewoldag
Copy link
Collaborator

Parsl supports caching of function output when properly configured, but this seems to be limited to computed output of a function. It may not immediately apply to our use case of files as output.

First we should investigate to determine if the above statement is true. Perhaps it does support caching of file output.

Second, we could implement a quick check in each function (though this should be pushed up to a parent class) that can look for the presence of a file on disk and short circuit the expensive work if the file already exists. Or, if a force=True parameter is passed, then redo the work to produce the output file.

@drewoldag drewoldag added the enhancement New feature or request label Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant