Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OSDF integration #15

Open
rkalyanapurdue opened this issue Apr 3, 2024 · 1 comment
Open

OSDF integration #15

rkalyanapurdue opened this issue Apr 3, 2024 · 1 comment
Assignees
@rkalyanapurdue
Labels
enhancement New feature or request

Comments

@rkalyanapurdue
Copy link
Collaborator

  1. Evaluate utilizing OSDF (https://osg-htc.org/services/osdf.html) to host large public datasets for use in I-GUIDE CI platform.
  2. Plan is to start with WRFHydro outputs from Anvil and move them to OSDF origin deployed on Purdue's CC* storage resource.
  3. Also evaluate and demonstrate Pelican project's fsspec Python library that can help slice NetCDF files similar to kerchunk.
@rkalyanapurdue rkalyanapurdue added the enhancement New feature or request label Apr 3, 2024
@rkalyanapurdue rkalyanapurdue self-assigned this Apr 3, 2024
@rkalyanapurdue
Copy link
Collaborator Author

Initial meeting held w/ Brian Bockelman from OSG group and Erik Gough from Purdue to discuss this integration. Next steps:

  1. Wait for StorageClass to be developed for integrating CC* storage with Geddes cloud.
  2. rsync example WRFHydro outputs from Anvil scratch to CC* storage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rkalyanapurdue rkalyanapurdue

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@rkalyanapurdue