Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

table_id replace null to "na" #101

Open
aradhakrishnanGFDL opened this issue Dec 12, 2024 · 2 comments
Open

table_id replace null to "na" #101

aradhakrishnanGFDL opened this issue Dec 12, 2024 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@aradhakrishnanGFDL
Copy link
Collaborator

Opening issue to reflect Ray's issues using the generated catalog in analysis.

It works sometimes, but segmentation faults at

d_ = catalog.search(**query_params).to_dataset_dict(progressbar=True)

I get that same warning. It comes from the fact that "table_id" can be either "fx" or blank, which pandas converts to NaN. So it's not sure if
table_id should be a numerical type or a string. Here is some output. The first time is succeeds, but the second time it fails:

(env) rlm:/local2/home/cleanup-fre-cli/tmp2> python3 catalog_test.py
/local2/home/cleanup-fre-cli/tmp2/env/lib/python3.10/site-packages/intake_esm/cat.py:249: DtypeWarning: Columns (6) have mixed types. Specify dtype option on import or set low_memory=False.
df = pd.read_csv(
(env) rlm:/local2/home/cleanup-fre-cli/tmp2> python3 catalog_test.py
/local2/home/cleanup-fre-cli/tmp2/env/lib/python3.10/site-packages/intake_esm/cat.py:249: DtypeWarning: Columns (6) have mixed types. Specify dtype option on import or set low_memory=False.
df = pd.read_csv(
Segmentation fault (core dumped)

@aradhakrishnanGFDL aradhakrishnanGFDL added the bug Something isn't working label Dec 12, 2024
@aradhakrishnanGFDL aradhakrishnanGFDL self-assigned this Dec 12, 2024
@ceblanton
Copy link
Contributor

Thanks Aparna! Interesting. Is the seg fault possibly unrelated to the table_id type issues?

Why would the seg fault not be repeatable?

@aradhakrishnanGFDL
Copy link
Collaborator Author

Not sure @ceblanton. In your catalog, you could change it to NA and see if Ray continues to see this issue or not to rule this out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants