Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add load-catalog cli command #45

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

simonwoerpel
Copy link
Contributor

This is a first lay out for a load-catalog command.

It accepts an url to a catalog index json and a few options (include/exclude specific datasets), creates or updates the collections metadata and bulk loads the entities into aleph.

A few questions arise:

  • should we just safely assume to map dataset names to aleph foreign_id ? This could write entities into existing collections with the same foreign_id that are originally not meant to be the dataset from the catalog
  • do we want more logic (e.g. skip importing for not updated since ...)
  • and something like "inspect catalog" to see some metadata without actually importing
  • how to handle collections/children?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant