-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feature(service): Adds checks for OSTI service
+ changes service.osti.submit to a private method + Adds checking to service.osti._submit for status and correct doi prefix, raises service exception if either of the checks fail. + Extends OSTIRecord dataclase to collect site_url and populates it from the osti response for a later check + Refactors service.osti.publish to check the site_url returned in the OSTIRecord, raises a ServiceAccountExcepton on failure. + Fixes logic error in service.osti.to_osti_xml where xml is prepared for publication. There was a possibility of creating a duplicate record or submitting for publication to osti before the dataset is approved. The first case can happen if a doi is created manually during draft and then the user clicks submit. The second case happens if the data is synchronized manually before approval. The solution is to check for the publication date and dataset.status == APPROVED before marking the dataset for publication. Additionally, set_reserved should only be set if dataset is None(dummy data) or there is no doi yet. Also, adds check if there is already a doi before adding set_reserved. + Updates existing tests for new logic. Close #405
- Loading branch information
Showing
7 changed files
with
192 additions
and
33 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <records> | ||
| <record status="Pending" released="N"> | ||
| <site_input_code>NGEE-TRPC</site_input_code> | ||
| <osti_id>1525114</osti_id> | ||
| <dataset_type>SM</dataset_type> | ||
| <title>Data Set 2</title> | ||
| <status>SUCCESS</status> | ||
| <authors> | ||
| <author> | ||
| <first_name>Luke</first_name> | ||
| <last_name>Cage</last_name> | ||
| <private_email>[email protected]</private_email> | ||
| <affiliation_name>POWER</affiliation_name> | ||
| </author> | ||
| </authors> | ||
| <related_resource></related_resource> | ||
| <product_nos>NGT0001</product_nos> | ||
| <product_type>DA</product_type> | ||
| <contract_nos>LBNL NGEE-Tropics UC, Berkeley NGEE-Tropics</contract_nos> | ||
| <other_identifying_numbers></other_identifying_numbers> | ||
| <originating_research_org>Next-Generation Ecosystem Experiments Tropics; LBNL</originating_research_org> | ||
| <availability></availability> | ||
| <collaboration_names></collaboration_names> | ||
| <publication_date>2016</publication_date> | ||
| <language>English</language> | ||
| <country>US</country> | ||
| <sponsor_org>A few funding organizations</sponsor_org> | ||
| <subject_categories_code>54 ENVIRONMENTAL SCIENCES</subject_categories_code> | ||
| <keywords></keywords> | ||
| <accession_num></accession_num> | ||
| <description>Qui illud verear persequeris te. Vis probo nihil verear an, zril tamquam philosophia eos te, quo ne fugit movet contentiones. Quas mucius detraxit vis an, vero omnesque petentium sit ea. Id ius inimicus comprehensam.</description> | ||
| <site_url>https://ngt-data.lbl.gov/dois/NGT0001</site_url> | ||
| <date_first_submitted>2022-03-24</date_first_submitted> | ||
| <date_last_submitted>2022-03-24</date_last_submitted> | ||
| <doi status="PENDING">10.15486/fob/1525114</doi> | ||
| <doi_infix>ngt</doi_infix> | ||
| <file_extension></file_extension> | ||
| <software_needed></software_needed> | ||
| <dataset_size></dataset_size> | ||
| <contact_name>NGEE Tropics Archive Team, Support Organization</contact_name> | ||
| <contact_org>Lawrence Berkeley National Lab</contact_org> | ||
| <contact_email>ngeet-team@testserver</contact_email> | ||
| <contact_phone>(510) 495-2905</contact_phone> | ||
| <othnondoe_contract_nos></othnondoe_contract_nos> | ||
| </record> | ||
| </records> |
47 changes: 47 additions & 0 deletions
47
archive_api/tests/osti_response_publish_error_site_url.xml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <records> | ||
| <record status="Pending" released="N"> | ||
| <site_input_code>NGEE-TRPC</site_input_code> | ||
| <osti_id>1525114</osti_id> | ||
| <dataset_type>SM</dataset_type> | ||
| <title>Data Set 2</title> | ||
| <status>SUCCESS</status> | ||
| <authors> | ||
| <author> | ||
| <first_name>Luke</first_name> | ||
| <last_name>Cage</last_name> | ||
| <private_email>[email protected]</private_email> | ||
| <affiliation_name>POWER</affiliation_name> | ||
| </author> | ||
| </authors> | ||
| <related_resource></related_resource> | ||
| <product_nos>NGT0001</product_nos> | ||
| <product_type>DA</product_type> | ||
| <contract_nos>LBNL NGEE-Tropics UC, Berkeley NGEE-Tropics</contract_nos> | ||
| <other_identifying_numbers></other_identifying_numbers> | ||
| <originating_research_org>Next-Generation Ecosystem Experiments Tropics; LBNL</originating_research_org> | ||
| <availability></availability> | ||
| <collaboration_names></collaboration_names> | ||
| <publication_date>2016</publication_date> | ||
| <language>English</language> | ||
| <country>US</country> | ||
| <sponsor_org>A few funding organizations</sponsor_org> | ||
| <subject_categories_code>54 ENVIRONMENTAL SCIENCES</subject_categories_code> | ||
| <keywords></keywords> | ||
| <accession_num></accession_num> | ||
| <description>Qui illud verear persequeris te. Vis probo nihil verear an, zril tamquam philosophia eos te, quo ne fugit movet contentiones. Quas mucius detraxit vis an, vero omnesque petentium sit ea. Id ius inimicus comprehensam.</description> | ||
| <site_url>https://vchendrix/dois/NGT0001</site_url> | ||
| <date_first_submitted>2022-03-24</date_first_submitted> | ||
| <date_last_submitted>2022-03-24</date_last_submitted> | ||
| <doi status="PENDING">10.15486/ngt/1525114</doi> | ||
| <doi_infix>ngt</doi_infix> | ||
| <file_extension></file_extension> | ||
| <software_needed></software_needed> | ||
| <dataset_size></dataset_size> | ||
| <contact_name>NGEE Tropics Archive Team, Support Organization</contact_name> | ||
| <contact_org>Lawrence Berkeley National Lab</contact_org> | ||
| <contact_email>ngeet-team@testserver</contact_email> | ||
| <contact_phone>(510) 495-2905</contact_phone> | ||
| <othnondoe_contract_nos></othnondoe_contract_nos> | ||
| </record> | ||
| </records> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,7 +9,7 @@ | |
| from archive_api.models import ServiceAccount | ||
|
|
||
|
|
||
| OSTI_XML = '<records><record><title>Data Set 2</title><contract_nos>LBNL NGEE-Tropics & UC, Berkeley NGEE-Tropics</contract_nos><non-doe_contract_nos>LBNL NGEE-Tropics & UC, Berkeley NGEE-Tropics</non-doe_contract_nos><originating_research_org>LBNL</originating_research_org><description>Qui illud verear persequeris te. Vis probo nihil verear an, zril tamquam philosophia eos te, quo ne fugit movet contentiones. Quas mucius detraxit vis an, vero omnesque petentium sit ea. Id ius inimicus comprehensam.</description><sponsor_org>A few funding organizations</sponsor_org><related_resource /><product_nos>NGT0001</product_nos><osti_id>892375dkfnsi</osti_id><site_url>https://ngt-data.lbl.gov/dois/NGT0001</site_url><publication_date>2016</publication_date><dataset_type>SM</dataset_type><contact_name>NGEE Tropics Archive Team, Support Organization</contact_name><contact_email>NGEE Tropics Archive Test <ngeet-team@testserver></contact_email><contact_org>Lawrence Berkeley National Lab</contact_org><site_code>NGEE-TRPC</site_code><doi_infix>ngt</doi_infix><subject_categories_code>54 ENVIRONMENTAL SCIENCES</subject_categories_code><language>English</language><country>US</country><creatorsblock><creators_detail><first_name>Luke</first_name><last_name>Cage</last_name><private_email>[email protected]</private_email><affiliation_name>POWER</affiliation_name></creators_detail></creatorsblock></record></records>' | ||
| OSTI_XML = '<records><record><title>Data Set 3</title><contract_nos>None</contract_nos><non-doe_contract_nos /><originating_research_org /><description>Qui illud verear persequeris te. Vis probo nihil verear an, zril tamquam philosophia eos te, quo ne fugit movet contentiones. Quas mucius detraxit vis an, vero omnesque petentium sit ea. Id ius inimicus comprehensam.</description><sponsor_org>A few funding organizations for my self</sponsor_org><related_resource /><product_nos>NGT0002</product_nos><osti_id>8343947</osti_id><site_url>https://ngt-data.lbl.gov/dois/NGT0002</site_url><publication_date>2016</publication_date><dataset_type>SM</dataset_type><contact_name>NGEE Tropics Archive Team, Support Organization</contact_name><contact_email>NGEE Tropics Archive Test <ngeet-team@testserver></contact_email><contact_org>Lawrence Berkeley National Lab</contact_org><site_code>NGEE-TRPC</site_code><doi_infix>ngt</doi_infix><subject_categories_code>54 ENVIRONMENTAL SCIENCES</subject_categories_code><language>English</language><country>US</country><creatorsblock><creators_detail><first_name>Luke</first_name><last_name>Cage</last_name><private_email>[email protected]</private_email><affiliation_name>POWER</affiliation_name></creators_detail></creatorsblock></record></records>' | ||
| OSTI_XML_DUMMY = '<records><record><title /><contract_nos>None</contract_nos><non-doe_contract_nos /><originating_research_org /><description /><sponsor_org /><related_resource /><product_nos /><set_reserved /><dataset_type>SM</dataset_type><contact_name>NGEE Tropics Archive Team, Support Organization</contact_name><contact_email>NGEE Tropics Archive Test <ngeet-team@testserver></contact_email><contact_org>Lawrence Berkeley National Lab</contact_org><site_code>NGEE-TRPC</site_code><doi_infix>ngt</doi_infix><subject_categories_code>54 ENVIRONMENTAL SCIENCES</subject_categories_code><language>English</language><country>US</country></record></records>' | ||
| BASEPATH = os.path.dirname(__file__) | ||
|
|
||
|
|
@@ -25,10 +25,11 @@ def django_load_data(django_db_setup, django_db_blocker): | |
| @pytest.mark.django_db | ||
| @pytest.mark.parametrize("dataset_id,expected_osti_xml", | ||
| [(None, OSTI_XML_DUMMY), | ||
| (2, OSTI_XML)], ids=["OSTI dummy", "OSTI publish"]) | ||
| (3, OSTI_XML)], ids=["OSTI-dummy", "OSTI-publish"]) | ||
| def test_to_osti(django_load_data, dataset_id, expected_osti_xml): | ||
| """Test the generation of OSTI dummy xml""" | ||
| osti_xml = to_osti_xml(dataset_id) | ||
| print(osti_xml) | ||
| assert osti_xml == expected_osti_xml | ||
|
|
||
|
|
||
|
|
@@ -63,28 +64,29 @@ def mock_post(*args, **kwargs): | |
| def test_osti_error(django_load_data, monkeypatch, status_code): | ||
| """Test publish""" | ||
|
|
||
| osti_response = open(os.path.join(BASEPATH, "osti_response_error.xml")).read() | ||
|
|
||
| def mock_post(*args, **kwargs): | ||
| assert "auth" in kwargs | ||
| assert kwargs["auth"] == ("myuseraccount", 'foobar') | ||
|
|
||
| return type('Dummy', (object,), { | ||
| "content": open(os.path.join(BASEPATH, "osti_response_error.xml")).read(), | ||
| "content": osti_response, | ||
| "status_code": status_code, | ||
| "url": "/testurl"}) | ||
|
|
||
| monkeypatch.setattr(requests, 'post', mock_post) | ||
|
|
||
| with pytest.raises(ServiceAccountException) as exec_info: | ||
| publish(2) | ||
|
|
||
| if status_code == 200: | ||
| osti_record = publish(2) | ||
| assert osti_record | ||
| assert osti_record.status == "FAILURE" | ||
| assert osti_record.doi_status is None | ||
| assert osti_record.doi is None | ||
| assert osti_record.status_message == ('Title is required.; DOE Contract/Award Number(s) is required.; Author(s) is ' | ||
| 'required.; Publication Date is required.; Sponsoring Organization is ' | ||
| 'required.; Missing required URL.') | ||
| assert str(exec_info.value) == ('Service Account OSTI Elink: Error in doi record - Title is required.; DOE ' | ||
| 'Contract/Award Number(s) is required.; Author(s) is required.; Publication ' | ||
| 'Date is required.; Sponsoring Organization is required.; Missing required ' | ||
| 'URL.') | ||
| else: | ||
| pytest.raises(ServiceAccountException, publish, 2) | ||
| assert str(exec_info.value) == 'Service Account OSTI Elink: HTTP Status: 500 HTTP Response: '+osti_response | ||
|
|
||
|
|
||
| @pytest.mark.django_db | ||
|
|
@@ -116,4 +118,38 @@ def mock_post(*args, **kwargs): | |
| @pytest.mark.django_db | ||
| def test_osti_mint_not_needed(django_load_data): | ||
| """Test osti mint returns None. The dataset already has a DOI """ | ||
| assert mint(2) is None | ||
| assert mint(2) is None | ||
|
|
||
|
|
||
|
|
||
| @pytest.mark.django_db | ||
| @pytest.mark.parametrize("osti_xml, error_message", | ||
| [("osti_response_publish_error_site_url.xml", | ||
| ('Service Account OSTI Elink: Error in publish data; There was a problem ' | ||
| 'minting the DOI - The site url, https://vchendrix/dois/NGT0001, does not ' | ||
| 'match our record, https://ngt-data.lbl.gov/dois/NGT0002.') | ||
| ), | ||
| ("osti_response_publish_error_doi.xml", | ||
| ('Service Account OSTI Elink: Error in publish data; There was a problem ' | ||
| 'minting the DOI - The doi, https://doi.org/10.15486/fob/1525114, is not a ' | ||
| 'valid doi'))], | ||
| ids=['site_url', 'doi']) | ||
| def test_osti_publish_errors(django_load_data, monkeypatch, osti_xml, error_message): | ||
| """Test publish errors""" | ||
|
|
||
| def mock_post(*args, **kwargs): | ||
| assert "auth" in kwargs | ||
| assert kwargs["auth"] == ("myuseraccount", 'foobar') | ||
|
|
||
| return type('Dummy', (object,), { | ||
| "content": open(os.path.join(BASEPATH, osti_xml)).read(), | ||
| "status_code": 200, | ||
| "url": "/testurl"}) | ||
|
|
||
| monkeypatch.setattr(requests, 'post', mock_post) | ||
|
|
||
| with pytest.raises(ServiceAccountException) as exec_info: | ||
| publish(3) | ||
|
|
||
|
|
||
| assert str(exec_info.value) == error_message | ||
Oops, something went wrong.