Connect IOOS (national) catalog into ODIS

[MathewBiddle]

IOOS would like to investigate how the IOOS Data Catalog could be indexed.

IOOS Data Catalog: https://data.ioos.us/
sitemap should be at https://data.ioos.us/sitemap.xml but needs to be fixed.

xref

• Investigate requirements for UN Decade ODIS/OIH interoperability for IOOS Catalog ioos/catalog#86

[jmckenna]

@MathewBiddle keep in mind also that we've been working with CIOOS for a while, and have worked through many issues together (they have a national CKAN instance for example, that federates 3 partner catalogues, so there could be a lot of similarities). Keep @fostermh & whole CIOOS team in mind, as you tackle sitemaps and CKAN extensions & templates (related to #321 )

[MathewBiddle]

@mwengren do we have an example dataset IOOS would like to test for this?

Maybe an NGDAC from ERDDAP dataset - https://data.ioos.us/dataset/?q=temperature&organization=glider-dac&res_format=ERDDAP-TableDAP

[MathewBiddle]

At a recent ESIP Marine Data Cluster meeting we (@pbuttigieg, @cberys, et al.) discussed how it would be helpful to do a live demonstration on how to connect a record into ODIS. We proposed to have this demonstration on February 29 3-4 ET. Is this something you would be interested in @jmckenna? If so, is there enough time/information to prepare for that?

[mwengren]

@MathewBiddle As far as the IOOS Catalog is concerned, it shouldn't matter too much in terms of the Schema.org metadata served (rules for generating it are the same for all datasets). But some datasets have better metadata than others.

Glider DAC files would probably be fine, but currently they don't include as many attributes from our IOOS Metadata Profile 1.2 guidance, so instead I'd go with one of the RA's datasets. Here are a few links including recent (time coverage from 2/1 to 2/15) datasets from some different RAs (any datasets shown below should be fine for testing purposes):

• GCOOS: https://data.ioos.us/dataset/?q=&sort=score+desc%2C+metadata_modified+desc&ext_timerange_start=2024-02-01T15%3A00Z&ext_timerange_end=2024-02-15T23%3A00Z&organization=gcoos
• CeNCOOS: https://data.ioos.us/dataset/?q=&sort=score+desc%2C+metadata_modified+desc&ext_timerange_start=2024-02-01T15%3A00Z&ext_timerange_end=2024-02-15T23%3A00Z&organization=cencoos
• PacIOOS: https://data.ioos.us/dataset/?q=&sort=score+desc%2C+metadata_modified+desc&ext_timerange_start=2024-02-01T15%3A00Z&ext_timerange_end=2024-02-15T23%3A00Z&organization=pacioos

PacIOOS custom-generates their own metadata, so they tend to be the most complete.

Here's a good example PaCIOOS dataset to test from the above link (this one has a point value for geographic extent, rather than a bounding box, which is a problem for many catalogs including CKAN that we had to write a workaround for - I'd be interested to see how ODIS handles it):

https://data.ioos.us/dataset/aloha-cabled-observatory-aco-acoustic-doppler-current-profiler-adcp-temperature.

[MathewBiddle]

The IOOS sitemap.xml is up and running again. https://data.ioos.us/sitemap.xml

We could also demo with the whole catalog? 32.5k records....

[r0sek]

At a recent ESIP Marine Data Cluster meeting we (@pbuttigieg, @cberys, et al.) discussed how it would be helpful to do a live demonstration on how to connect a record into ODIS. We proposed to have this demonstration on February 29 3-4 ET. Is this something you would be interested in @jmckenna? If so, is there enough time/information to prepare for that?

@MathewBiddle is this demo confirmed?

[pbuttigieg]

@r0sek it should be in the ESIP calendar for the 29th soon. I believe the time is 20:00 UTC

[pbuttigieg]

@MathewBiddle and team put up this IOOS JSON-LD (trom CKAN) for a test conversion:
https://data.ioos.us/dataset/44cc60f9-376a-4090-b220-ede03f31dce9.jsonld

@jmckenna this is pretty much the same scenario as that in the CIOOS and PDH plugins. We'll need a little JSON framing and context conversion to schema.org, but it looks quite straightforward.

The next steps are for us to work with @MathewBiddle to create a couple of records in the vanilla form for the IOOS team to use as references during an upcoming hackathon.

[MathewBiddle]

for reference, CIOOS json-ld framing script - https://github.com/iodepo/odis-arch/blob/master/utils/jsonldProcessing/framing/framing-json-ld.py

[pbuttigieg]

for reference, CIOOS json-ld framing script - https://github.com/iodepo/odis-arch/blob/master/utils/jsonldProcessing/framing/framing-json-ld.py

You can also experiment in the JSON-LD Playground.

[MathewBiddle]

@mwengren FYI, I went ahead and registered the IOOS Data Catalog in ODISCat https://catalogue.odis.org/view/3308. It doesn't look like the sitemap is being read, but I wanted to see how broken things would get right out of the box.

[MathewBiddle]

Seems that the record is online now:

[MathewBiddle]

Now that it's online, I'm curious what's working and what's not. I don't see IOOS in http://dashboard.oceaninfohub.org/

[pbuttigieg]

@fils @jmckenna I'm assuming this is waiting for the automation deployment?

[GabCan]

@pbuttigieg are there additional steps we need to take from our side?