Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Submission Summary table in RODA Clinvar dataset not queryable in Athena #27

Open
budsonjelmont opened this issue Jul 31, 2023 · 0 comments

Comments

@budsonjelmont
Copy link

budsonjelmont commented Jul 31, 2023

I'm trying to demo Athena using the pre-built RODA Clinvar dataset. I deployed using the stack here: https://github.com/aws-samples/data-lake-as-code/tree/roda#readme.

The stack deploys without any issues noted, and after that I'm able to query most of the tables from Athena. However, when I run the following command in Athena:

SELECT * FROM "clinvar_summary_variants_dl-awsroda"."submission_summary";

The count of records returned is as-expected, but the values in every column are empty.

Notably, there appears to be a disconnect between the column names in the glue data catalog and the column names in the corresponding parquet metadata: in the data catalog columns are assigned distinct names (e.g. variationid, clinicalsignificance,...) in accordance with the official clinvar file definition README, but in the parquet file for this table each column appears to be named 'col' + autoincremented integer (e.g. col0, col1,...).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant