DC-1238: Add dummy genomic data to test OMOP datasets

[pshapiro4broad]

Jira ticket: https://broadworkbench.atlassian.net/browse/DC-1238

Addresses

Update test OMOP data for integration / connected tests and for local dataset setup to include a sample table with dummy genomic data.

Summary of changes

• add Sample table to schema and settings for test data
• minor changes to setup_tdr_resources.py
• rename a .jsonl file to .json

Testing Strategy

• integration / connected tests
• ran setup_tdr_resources.py locally

[s-rubenstein]

Looks reasonable

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[fboulnois]

See below:

[fboulnois]

this looks like a jsonl?

[pshapiro4broad]

Yes, all the data tables are jsonl. I found a number of changes like this that would be an improvement, but decided to limit the work done in this PR, I'll make another tech debt ticket to further clean up this code

[fboulnois]

The file extension is json, should it be jsonl?

[rushtong]

Looks reasonable to me. My nagging concern here would be the tight coupling between the test classes (integration, service, and DAO tests) to the single dataset file which is how these tests were originally written.

[pshapiro4broad]

@rushtong

My nagging concern here would be the tight coupling between the test classes (integration, service, and DAO tests) to the single dataset file which is how these tests were originally written.

Is your concern about having multiple tests depend on the same files, or having tests depend on files outside the test itself, or something else?

For tests that require a lot of assets to set up, I think it makes sense to use external data files like this, rather than embedding the json in the test code, or adding objects that define the models for the json data. But I agree that this separation means that you have to be extra careful when updating the data files.

[rjohanek]

this looks good! are the datasets in dev/prod going to be updated as a part of this work? I'm excited to see the new table in action

[rushtong]

@pshapiro4broad

Is your concern about having multiple tests depend on the same files, or having tests depend on files outside the test itself, or something else?

For tests that require a lot of assets to set up, I think it makes sense to use external data files like this, rather than embedding the json in the test code, or adding objects that define the models for the json data. But I agree that this separation means that you have to be extra careful when updating the data files.

My concern is that each of the different test classes are now dependent on the same underlying data such that any change to the underlying data affects multiple tests in potentially different ways - essentially the last point that you make.