Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added grib idx extract methods and test cases #484

Merged
merged 7 commits into from
Aug 30, 2024

Conversation

Anu-Ra-g
Copy link
Contributor

@Anu-Ra-g Anu-Ra-g commented Jul 25, 2024

The extract_datatree_chunk_index function is essential for mapping the idx and grib metadata to make a mapping from one to the other for a particular model horizon file. Also, the mapping method inside the workflow uses the extract_datatree_chunk_index internally.

@martindurant martindurant changed the title added the extract methods and test cases added grib idx extract methods and test cases Jul 25, 2024
@martindurant
Copy link
Member

Sorry it has taken me so long to start looking at this.

Can you please provide a description for this PR, what the two main functions here are for, how they are expected to be used?

@Anu-Ra-g
Copy link
Contributor Author

Anu-Ra-g commented Aug 4, 2024

The extract_datatree_chunk_index function travel through each of the node in the datatree and extracts the grib metadata and returns it in the form of a Dataframe.

dt_instance = datatree.open_datatree(...)
extract_datatree_chunk_index(dt_instance, kerchunk_store)

@martindurant
Copy link
Member

Yes, but why, what are you supposed to do with this new code?

No description provided.

@Anu-Ra-g
Copy link
Contributor Author

Anu-Ra-g commented Aug 5, 2024

The dataframe from this function and the dataframe from the parse_grib_idx function are used in the one-to-one mapping. This mapping provides a aggregrate view of the dataset.

@martindurant
Copy link
Member

Please fill out the description of this PR at the top, saying why this new code should be included.

This mapping provides a aggregrate view of the dataset.

Please demonstrate how the code is supposed to be used.

@martindurant martindurant merged commit eaae20f into fsspec:main Aug 30, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants