Skip to content

Data integration steps

Jump to bottom
Daniel Santillan edited this page Jul 19, 2024 · 5 revisions

In this page we try to cover potential data sources and what options for integration can be explored independently. Following sections cover different ways that data is provided.

Data is provided as service

If the service is supported as described in the Resource section a branch can be created in the catalog repository adding a yaml file as described in the wiki. Assets linked in the yaml file need to be added to the assets repository.

As quick non extensive reference here is a list with examples yaml files:

Data is provided raw

If the data is not provided through a service but is accessible publicly and is small or in cloud optimized format and allows cross origin requests, integration can potentially be done by EOX and can be discussed. If one of these points does not apply the data needs to be copied (potentially reprocessed) and hosted through a supported service.

Integration testing

In order to independently test data integration following steps can be done.

  1. Create a branch in the catalog repository where the new collection yaml has been added
  2. Add the collection file name (without extension) to the corresponding catalog definition
  3. Make sure any referenced assets have been added to the assets repository
  4. Create a pull request for the branch where the pull request text is (a space separated list) of the filename(s) (without extension) for the collections to be tested.
  5. Check the automatic action was triggered and runs through successfully (it is referenced in the pull request but can also be found in the Actions page.
  6. Go to the staging or testing dashboard instance where you want to test the integration and add ?catalog=<branch_name> to the url. <branch_name> being the name you gave to the branch. For example https://testing-instance.eox.at/?catalog=integrationtest.
  7. If all looks as expected request EOX to merge the pull request
Clone this wiki locally