Skip to content

Commit

Permalink
Merge pull request #381 from gaiaresources/BDRSPS-1152-reusable-site-…
Browse files Browse the repository at this point in the history
…mapping

BDRSPS-1152 RDF mapping changes for re-usable Sites
  • Loading branch information
chungvl authored Jan 15, 2025
2 parents 107a014 + 23f8a3a commit 04aebd9
Show file tree
Hide file tree
Showing 15 changed files with 267 additions and 105 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -542,7 +542,7 @@ def apply_mapping_row(
# Create Tern Site IRI, depending on the siteID field
site_id: str | None = row["siteID"]
if site_id:
site = utils.iri_patterns.site_iri(base_iri, site_id)
site = utils.iri_patterns.legacy_site_iri(base_iri, site_id)
else:
site = None

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,5 @@ providerRecordID,providerRecordIDSource,locality,decimalLatitude,decimalLongitud
12,WAM,,-33.8,115.21,WGS84,,,26/09/2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Caladenia excelsa,,,,Plantae,,,,,,,,,,,,,
13,WAM,Cowaramup Bay Road,-33.86,115.02,WGS84,,,26/09/2019,,,PreservedSpecimen,Stream Environment and Water Pty Ltd,,,,,,,,,,,,,,C01,CC123,WAM,,,,,,,,,Stream Environment and Water Pty Ltd,,Caladenia excelsa,,,,Plantae,,,,,,,,,,,,,
14,WAM,Cowaramup Bay Road,-33.86,115.02,WGS84,20,Coordinates rounded to the nearest 10 km for conservation concern,26/09/2019,,,HumanObservation,Stream Environment and Water Pty Ltd,,,,,,,,,,,,,,C01,CC456,WAM,,,,,,,Caladenia ?excelsa,,Stream Environment and Water Pty Ltd,,Caladenia excelsa,?,Could not confirm the ID due to damaged flower,,Plantae,,,,,,,,,,,,,
8022FSJMJ079c5cf,WAM,Cowaramup Bay Road,-33.8,115.21,WGS84,50,Coordinates rounded to the nearest 10 km for conservation concern,26/09/2019,,human observation,PreservedSpecimen,Stream Environment and Water Pty Ltd,PE:12:8832,present,"Closed forest of Melaleuca lanceolata. White, grey or brown sand, sandy loam.",native,Dried out leaf tips,2,,,adult,male,No breeding evident,MR-456,Stream Environment and Water Pty Ltd,32237,ARACH,WAM,BHP2012-7521 | M12378,BHP,Wet (in ethanol or some other preservative),26/09/2019,https://www.ncbi.nlm.nih.gov/nuccore/MH040669.1 | https://www.ncbi.nlm.nih.gov/nuccore/MH040616.1,Sanger dideoxy sequencing,Caladenia ?excelsa,2019-09-27T12:34+08:00,Stream Environment and Water Pty Ltd,Visually identified in the field (sighting),Caladenia excelsa,species incerta,no flowers present,Caladenia excelsa Hopper & A.P.Br.,Plantae,species,VU,WA,Check against Threatened and Priority Fauna List WA available from https://www.dpaw.wa.gov.au/plants-and-animals/threatened-species-and-communities/threatened-animals. Last updated 13 June 2022,,WA-BIO,Category 1,Department of Biodiversity and Conservation,MR-R1,MR-S1,WAM,,MR-R1-V1
8022FSJMJ079c5cf,WAM,Cowaramup Bay Road,-33.8,115.21,WGS84,50,Coordinates rounded to the nearest 10 km for conservation concern,26/09/2019,,human observation,PreservedSpecimen,Stream Environment and Water Pty Ltd,PE:12:8832,present,"Closed forest of Melaleuca lanceolata. White, grey or brown sand, sandy loam.",native,Dried out leaf tips,2,,,adult,male,No breeding evident,MR-456,Stream Environment and Water Pty Ltd,32237,ARACH,WAM,BHP2012-7521 | M12378,BHP,Wet (in ethanol or some other preservative),26/09/2019,https://www.ncbi.nlm.nih.gov/nuccore/MH040669.1 | https://www.ncbi.nlm.nih.gov/nuccore/MH040616.1,Sanger dideoxy sequencing,Caladenia ?excelsa,2019-09-27T12:34+08:00,Stream Environment and Water Pty Ltd,Visually identified in the field (sighting),Caladenia excelsa,species incerta,no flowers present,Caladenia excelsa Hopper & A.P.Br.,Plantae,species,VU,WA,Check against Threatened and Priority Fauna List WA available from https://www.dpaw.wa.gov.au/plants-and-animals/threatened-species-and-communities/threatened-animals. Last updated 13 June 2022,,WA-BIO,Category 1,Department of Biodiversity and Conservation,MR-R1,MR-S1,WAM,https://example.com/site/WAM-MR-S1,MR-R1-V1
ABC123,WAM,Cowaramup Bay Road,-33.8,115.21,WGS84,30,Coordinates generalised,26/09/2019,,new sampling protocol,new basis of record,Stream Environment and Water Pty Ltd,PE:12:8833,new occurrence status,new habitat,new establishment means,Leaves brown,6,,,new life stage,new sex,new reproductiveCondition,MR-457,Stream Environment and Water Pty Ltd,32238,ARACH,WAM,BHP2012-7522 | M12379,BHP,new preparations,27/09/2019,https://www.ncbi.nlm.nih.gov/nuccore/MH040669.1 | https://www.ncbi.nlm.nih.gov/nuccore/MH040616.1,new sequencing method,Caladenia ?excelsa,2019-09-27T12:34+08:00,Stream Environment and Water Pty Ltd,new identification method,Caladenia excelsa,new identification qualifier,new remarks,Caladenia excelsa Hopper & A.P.Br.,new kingdom,new taxon rank,new threat status,WA,a random selection,,,Category 1,Department of Biodiversity and Conservation,MR-R1,MR-S1,WAM,,
Original file line number Diff line number Diff line change
Expand Up @@ -770,6 +770,16 @@
skos:prefLabel "Stream Environment and Water Pty Ltd recordNumber" ;
prov:wasAttributedTo <https://linked.data.gov.au/dataset/bdr/org/Stream-Environment-and-Water-Pty-Ltd> .

<https://linked.data.gov.au/dataset/bdr/datatypes/siteID/WAM> a rdfs:Datatype ;
skos:definition "An identifier for the site" ;
skos:prefLabel "WAM Site ID" ;
prov:qualifiedAttribution <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/attribution/WAM/resourceProvider> .

<https://example.com/site/WAM-MR-S1> a tern:FeatureOfInterest,
tern:Site ;
schema:identifier "MR-S1"^^<https://linked.data.gov.au/dataset/bdr/datatypes/siteID/WAM> ;
tern:featureType <http://linked.data.gov.au/def/tern-cv/5bf7ae21-a454-440b-bdd7-f2fe982d8de4> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/SiteVisit/MR-R1-V1> a tern:SiteVisit ;
schema:isPartOf <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000> .

Expand Down Expand Up @@ -897,10 +907,6 @@
prov:agent <https://linked.data.gov.au/dataset/bdr/org/Stream-Environment-and-Water-Pty-Ltd> ;
prov:hadRole <https://linked.data.gov.au/def/data-roles/owner> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/attribution/WAM/resourceProvider> a prov:Attribution ;
prov:agent <https://linked.data.gov.au/dataset/bdr/org/WAM> ;
prov:hadRole <https://linked.data.gov.au/def/data-roles/resourceProvider> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/bdr-cv/attribute/basisOfRecord/new-basis-of-record> a skos:Concept ;
skos:broader <http://example.com/bdr-cv/attribute/basisOfRecord> ;
skos:definition "A type of basisOfRecord." ;
Expand Down Expand Up @@ -1309,14 +1315,17 @@
<https://linked.data.gov.au/dataset/bdr/org/WA-BIO> a prov:Agent ;
schema:name "WA-BIO" .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/Site/MR-S1> a tern:FeatureOfInterest,
<https://linked.data.gov.au/dataset/bdr/site/WAM/MR-S1> a tern:FeatureOfInterest,
tern:Site ;
schema:isPartOf <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000> ;
tern:featureType <http://linked.data.gov.au/def/tern-cv/5bf7ae21-a454-440b-bdd7-f2fe982d8de4> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/Survey/MR-R1> a tern:Survey ;
schema:isPartOf <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/attribution/WAM/resourceProvider> a prov:Attribution ;
prov:agent <https://linked.data.gov.au/dataset/bdr/org/WAM> ;
prov:hadRole <https://linked.data.gov.au/def/data-roles/resourceProvider> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/bdr-cv/methods/identificationMethod/Visually-identified-in-the-field-sighting> a skos:Concept ;
skos:broader <http://example.com/bdr-cv/methods/identificationMethod> ;
skos:definition "A type of identificationMethod." ;
Expand Down Expand Up @@ -1689,7 +1698,7 @@
dwc:catalogNumber "ARACH"^^<https://linked.data.gov.au/dataset/bdr/datatypes/catalogNumber/WAM> ;
dwc:collectionCode "32237" ;
prov:wasAssociatedWith <https://linked.data.gov.au/dataset/bdr/org/Stream-Environment-and-Water-Pty-Ltd> ;
sosa:isSampleOf <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/Site/MR-S1> ;
sosa:isSampleOf <https://example.com/site/WAM-MR-S1> ;
sosa:usedProcedure <http://linked.data.gov.au/def/tern-cv/ea1d6342-1901-4f88-8482-3111286ec157> ;
schema:identifier "BHP2012-7521"^^<https://linked.data.gov.au/dataset/bdr/datatypes/catalogNumber/BHP>,
"M12378"^^<https://linked.data.gov.au/dataset/bdr/datatypes/catalogNumber/BHP>,
Expand All @@ -1710,7 +1719,7 @@
dwc:catalogNumber "ARACH"^^<https://linked.data.gov.au/dataset/bdr/datatypes/catalogNumber/WAM> ;
dwc:collectionCode "32238" ;
prov:wasAssociatedWith <https://linked.data.gov.au/dataset/bdr/org/Stream-Environment-and-Water-Pty-Ltd> ;
sosa:isSampleOf <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/Site/MR-S1> ;
sosa:isSampleOf <https://linked.data.gov.au/dataset/bdr/site/WAM/MR-S1> ;
sosa:usedProcedure <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/bdr-cv/methods/samplingProtocol/new-sampling-protocol> ;
schema:identifier "BHP2012-7522"^^<https://linked.data.gov.au/dataset/bdr/datatypes/catalogNumber/BHP>,
"M12379"^^<https://linked.data.gov.au/dataset/bdr/datatypes/catalogNumber/BHP>,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,6 @@
skos:prefLabel "Gaia Resources recordID" ;
prov:qualifiedAttribution <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/attribution/Gaia-Resources/resourceProvider> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/Site/P1> a tern:FeatureOfInterest,
tern:Site ;
schema:isPartOf <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000> ;
tern:featureType <http://linked.data.gov.au/def/tern-cv/5bf7ae21-a454-440b-bdd7-f2fe982d8de4> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/attribution/Gaia-Resources/resourceProvider> a prov:Attribution ;
prov:agent <https://linked.data.gov.au/dataset/bdr/org/Gaia-Resources> ;
prov:hadRole <https://linked.data.gov.au/def/data-roles/resourceProvider> .
Expand All @@ -72,9 +67,13 @@
<https://linked.data.gov.au/dataset/bdr/org/Gaia-Resources> a prov:Agent ;
schema:name "Gaia Resources" .

<https://linked.data.gov.au/dataset/bdr/site/TERN/P1> a tern:FeatureOfInterest,
tern:Site ;
tern:featureType <http://linked.data.gov.au/def/tern-cv/5bf7ae21-a454-440b-bdd7-f2fe982d8de4> .

<https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/occurrence/A0010> a dwc:Occurrence,
tern:FeatureOfInterest ;
sosa:isSampleOf <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000/Site/P1> ;
sosa:isSampleOf <https://linked.data.gov.au/dataset/bdr/site/TERN/P1> ;
sosa:usedProcedure <https://linked.data.gov.au/def/nrm/a8db263e-7a39-5b85-a5d2-8e1af7ce1e39> ;
schema:isPartOf <https://linked.data.gov.au/dataset/bdr/00000000-0000-0000-0000-000000000000> ;
schema:spatial _:Nb0c3d4fa822b88b4d3f8743700000000 ;
Expand Down
108 changes: 101 additions & 7 deletions abis_mapping/templates/survey_occurrence_data_v3/mapping.py
Original file line number Diff line number Diff line change
Expand Up @@ -561,13 +561,30 @@ def apply_mapping_row(
else:
survey = None

# Create Tern Site IRI, depending on the siteID field
# Create Tern Site IRI, depending on the site fields
site_id: str | None = row["siteID"]
if site_id:
site = utils.iri_patterns.site_iri(base_iri, site_id)
site_id_source: str | None = row["siteIDSource"]
existing_site_iri: str | None = row["existingBDRSiteIRI"]
if existing_site_iri:
site = rdflib.URIRef(existing_site_iri)
elif site_id and site_id_source:
site = utils.iri_patterns.site_iri(site_id_source, site_id)
else:
site = None

# When both existingBDRSiteIRI and siteID+siteIDSource are provided,
# the site gets a schema:identifier with this datatype.
if existing_site_iri and site_id and site_id_source:
site_id_datatype = utils.iri_patterns.datatype_iri("siteID", site_id_source)
site_id_datatype_attribution = utils.iri_patterns.attribution_iri(
base_iri, "resourceProvider", site_id_source
)
site_id_datatype_agent = utils.iri_patterns.agent_iri("org", site_id_source)
else:
site_id_datatype = None
site_id_datatype_attribution = None
site_id_datatype_agent = None

# Conditionally create uri dependent on siteVisitID field.
site_visit_id: str | None = row["siteVisitID"]
if site_visit_id:
Expand Down Expand Up @@ -1207,7 +1224,26 @@ def apply_mapping_row(
# Add site
self.add_site(
uri=site,
dataset=dataset,
site_id=site_id,
site_id_datatype=site_id_datatype,
graph=graph,
)

self.add_site_id_datatype(
uri=site_id_datatype,
site_id_source=site_id_source,
site_id_datatype_attribution=site_id_datatype_attribution,
graph=graph,
)
self.add_attribution(
uri=site_id_datatype_attribution,
provider=site_id_datatype_agent,
provider_role_type=DATA_ROLE_RESOURCE_PROVIDER,
graph=graph,
)
self.add_site_id_datatype_agent(
uri=site_id_datatype_agent,
site_id_source=site_id_source,
graph=graph,
)

Expand Down Expand Up @@ -4116,15 +4152,18 @@ def add_organism_quantity_value(

def add_site(
self,
*,
uri: rdflib.URIRef | None,
dataset: rdflib.URIRef,
site_id: str | None,
site_id_datatype: rdflib.URIRef | None,
graph: rdflib.Graph,
) -> None:
"""Adds site to the graph.
Args:
uri (rdflib.URIRef | None): URI to use if site provided else None.
dataset (rdflib.URIRef): The dataset which the data belongs.
site_id: Value of siteID field from the Row.
site_id_datatype: Datatype to use for the site id literal.
graph (rdflib.URIRef): Graph to be modified.
"""
# Check site uri exists
Expand All @@ -4134,9 +4173,64 @@ def add_site(
# Add site information to graph
graph.add((uri, a, utils.namespaces.TERN.Site))
graph.add((uri, a, utils.namespaces.TERN.FeatureOfInterest))
graph.add((uri, rdflib.SDO.isPartOf, dataset))
graph.add((uri, utils.namespaces.TERN.featureType, vocabs.site_type.SITE.iri))

if site_id_datatype is not None and site_id:
graph.add((uri, rdflib.SDO.identifier, rdflib.Literal(site_id, datatype=site_id_datatype)))

def add_site_id_datatype(
self,
*,
uri: rdflib.URIRef | None,
site_id_source: str | None,
site_id_datatype_attribution: rdflib.URIRef | None,
graph: rdflib.Graph,
) -> None:
"""Adds site id datatype to the graph.
Args:
uri: Subject of the node.
site_id_source: The siteIDSource value from the row.
site_id_datatype_attribution: The datatype attribution node.
graph: Graph to be modified.
"""
# Check subject was provided
if uri is None:
return
# Add type
graph.add((uri, a, rdflib.RDFS.Datatype))
# Add definition
graph.add((uri, rdflib.SKOS.definition, rdflib.Literal("An identifier for the site")))
# Add label
if site_id_source:
graph.add((uri, rdflib.SKOS.prefLabel, rdflib.Literal(f"{site_id_source} Site ID")))
# Add attribution link
if site_id_datatype_attribution:
graph.add((uri, rdflib.PROV.qualifiedAttribution, site_id_datatype_attribution))

def add_site_id_datatype_agent(
self,
*,
uri: rdflib.URIRef | None,
site_id_source: str | None,
graph: rdflib.Graph,
) -> None:
"""Adds the site id datatype agent to the graph.
Args:
uri: Subject of the node.
site_id_source: The siteIDSource value from the row.
graph: Graph to be modified.
"""
# Check subject provided
if uri is None:
return
# Add type
graph.add((uri, a, rdflib.PROV.Agent))
# Add name
if site_id_source:
graph.add((uri, rdflib.SDO.name, rdflib.Literal(site_id_source)))

def add_sensitivity_category_attribute(
self,
uri: rdflib.URIRef | None,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ bdrsh:DatatypeShape
a rdfs:Class, sh:NodeShape ;
sh:targetClass rdfs:Datatype ;
sh:xone (
bdrsh:SiteIDDatatypeShape
bdrsh:RecordIDDatatypeShape
bdrsh:RecordNumberDatatypeShape
bdrsh:CatalogNumberDatatypeShape
Expand Down Expand Up @@ -92,6 +93,32 @@ bdrsh:RecordIDDatatypeShape
sh:class prov:Attribution ;
] .

bdrsh:SiteIDDatatypeShape
a sh:NodeShape ;
sh:property [
sh:path skos:prefLabel ;
sh:datatype xsd:string ;
sh:nodeKind sh:Literal ;
sh:minCount 1 ;
sh:maxCount 1 ;
sh:pattern " Site ID$" ;
] ;
sh:property [
sh:path skos:definition ;
sh:datatype xsd:string ;
sh:nodeKind sh:Literal ;
sh:minCount 1 ;
sh:maxCount 1 ;
sh:pattern "^An identifier for the site$" ;
] ;
sh:property [
sh:path prov:qualifiedAttribution ;
sh:minCount 1 ;
sh:maxCount 1 ;
sh:node bdrsh:AttributionShape ;
sh:class prov:Attribution ;
] .

bdrsh:AttributionShape
a sh:NodeShape ;
sh:property [
Expand Down
4 changes: 2 additions & 2 deletions abis_mapping/templates/survey_site_data_v2/mapping.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ def apply_mapping_row(
# TERN.Site subject IRI - Note this needs to match the iri construction of the
# survey site visit and occurrence template mapping, ensuring they will resolve properly.
site_id: str = row["siteID"]
site = utils.iri_patterns.site_iri(base_iri, site_id)
site = utils.iri_patterns.legacy_site_iri(base_iri, site_id)

# Conditionally create uris dependent on siteIDSource
site_id_src: str | None = row["siteIDSource"]
Expand All @@ -258,7 +258,7 @@ def apply_mapping_row(
rtor_site_vocab = self.fields()["relationshipToRelatedSite"].get_vocab()
if rtor_site_vocab().get(relationship_to_related_site) == rdflib.SDO.isPartOf:
# Related site is defined internal to the dataset
related_site = utils.iri_patterns.site_iri(base_iri, related_site_id)
related_site = utils.iri_patterns.legacy_site_iri(base_iri, related_site_id)
else:
# Related site is defined outside the dataset
related_site = utils.rdf.uri_or_string_literal(related_site_id)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
siteID,siteIDSource,existingBDRSiteIRI,siteType,siteName,siteDescription,habitat,relatedSiteID,relationshipToRelatedSite,locality,decimalLatitude,decimalLongitude,footprintWKT,geodeticDatum,coordinateUncertaintyInMeters,dataGeneralizations
P0,WAM,,Site,ParentSite,Footprint of study area,Closed forest,,,Cowaramup Bay Road,,,"POLYGON ((114.98 -33.85, 115.01 -33.85, 115.01 -33.87, 114.98 -33.87, 114.98 -33.85))",WGS84,50,
P1,WAM,,Plot,Plot 1,Fine woody debris.,Closed forest,P0,partOf,Cowaramup Bay Road,-33.85,114.99,"LINESTRING (114.99 -33.85, 115.00 -33.85)",WGS84,50,Coordinates rounded to the nearest 10 km for conservation concern
P0,WAM,https://example.com/site/WAM/P0,Site,ParentSite,Footprint of study area,Closed forest,,,Cowaramup Bay Road,,,"POLYGON ((114.98 -33.85, 115.01 -33.85, 115.01 -33.87, 114.98 -33.87, 114.98 -33.85))",WGS84,50,
,,https://example.com/site/WAM/P1,Plot,Plot 1,Fine woody debris.,Closed forest,P0,partOf,Cowaramup Bay Road,-33.85,114.99,"LINESTRING (114.99 -33.85, 115.00 -33.85)",WGS84,50,Coordinates rounded to the nearest 10 km for conservation concern
P2,WAM,,Plot,Plot 2,Fine woody debris.,Closed forest,S0,sameAs,Cowaramup Bay Road,-33.85,114.99,"LINESTRING (114.99 -33.85, 115.00 -33.85)",WGS84,50,Coordinates rounded to the nearest 10 km for conservation concern
P3,WAM,,Plot,Plot 3,Fine woody debris.,Closed forest,http://example.com/site/S0,sameAs,Cowaramup Bay Road,-33.85,114.99,"LINESTRING (114.99 -33.85, 115.00 -33.85)",WGS84,50,Coordinates rounded to the nearest 10 km for conservation concern
Loading

0 comments on commit 04aebd9

Please sign in to comment.