Ckan Datasource

CKAN is the world’s leading open-source data portal platform. CKAN makes it easy to publish, share and work with data. It's a data management system that provides a powerful platform for cataloging, storing and accessing datasets with a rich front-end, full API (for both data and catalog), visualization tools and more.

The DVF module supports visualising data from a CKAN datasource. To do this, you require the CKAN Connector (ckan_connect) companion module

Getting started

Install ckan_connect with composer

composer require drupal/ckan_connect

Then enable ckan_connect and dvf_ckan via drupal UI or drush en dvf_ckan ckan_connect

Configuring

Before you can connect to you need to add your CKAN instance API URL and API KEY to CKAN Connect settings in Drupal, visit /admin/config/services/ckan-connect (Configuration > Web services > CKAN Connect). The API URL generally be CKAN_URL:CKAN_PORT/api/3. The API KEY is optional and only required if you are accessing private records.

Don't have a CKAN datasource?

If you would like to test out the CKAN datasource functionality but don't have a CKAN instance to connect to, you can easily spin up your own CKAN instance using docker. As CKAN requires a few services to get up and running with full functionality, we recommend using a pre-made stack. The okfn/docker-ckan repo is well maintained and documented, it should provide you a working CKAN instance which you can test with DVF.

This guide will step you through the minimum steps to get your own CKAN instance up and working with DVF. Ensure you have docker and docker-compose installed before continuing.

Installation

The okfn/docker-ckan README has more information on this, but essentially the steps are, clone the repo, set env variables, build then run.

Clone

git clone [email protected]:okfn/docker-ckan.git && cd docker-ckan

Environment variables

cp .env.example .env && nano .env

Get the IP address of your docker host (if running locally this will be the IP address of your computer). Edit the .env file and update the following variables with your IP: CKAN_SITE_URL, CKAN_DATAPUSHER_URL and CKAN__DATAPUSHER__CALLBACK_URL_BASE. It is best not to use localhost as this may cause issues when attempting to connect to it using dvf_ckan

Eg if your IP is 192.168.0.10 make these changes:

CKAN_SITE_URL=http://ckan:5000  >  CKAN_SITE_URL=http://192.168.0.10:5000
CKAN_DATAPUSHER_URL=http://datapusher:8800  >  CKAN_DATAPUSHER_URL=http://192.168.0.10:8800
CKAN__DATAPUSHER__CALLBACK_URL_BASE=http://ckan:5000  >  CKAN__DATAPUSHER__CALLBACK_URL_BASE=http://192.168.0.10:5000

Build and run

docker-compose build
docker-compose up -d

Assign superpowers - The ckan_admin user is initially created as a basic user, to perform administration tasks we need to make them a CKAN sysadmin. Run the following:

docker-compose exec ckan ckan -c ckan.ini sysadmin add ckan_admin

NOTE: there a re many more ckan console commands available, older guides will often reference paster, in this environment you can substitute these via docker-compose exec ckan ckan -c ckan.ini COMMAND

Access the UI - via the CKAN_SITE_URL you defined in .env. Eg visit http://192.168.0.10:5000. You should see a CKAN homepage. You can then login, the default credentials are defined in .env as User ckan_admin Pass test1234

Adding your first dataset

The full documentation for using CKAN can be found here but this guide will just focus on the minimum required to expose a dataset so you can use it as a DVF source.

Steps

• Create an organisation - Click Organisations link then click, "Add Organisation", give it a name then save.
• Add a dataset - From your organisation page, click "Add Dataset"
  • Give the dataset a name, set visibility to "public" then click "Next: Add Data"
  • Upload a CSV and set format to CSV
  • Click finish
• Click "Explore" > "Edit"
  • Click "Datastore" tab
  • Click "Upload to datastore" and refresh every few seconds until "Status" = "Complete"
• Test the API is working
  • View your resource, and copy the resource id from the URL, eg 3220de3f-9f1d-498e-95eb-140544981a42
  • Visit the api url for datastore for this resource http://YOUR_IP:5000/api/3/action/datastore_search?id=3220de3f-9f1d-498e-95eb-140544981a42
  • You should see the data from your CSV, this means the API and datapusher are functioning correctly
  • Copy the resource URL from CKAN eg http://YOUR_IP:5000/dataset/NAME/resource/RESOURCE_ID (get this by visiting a resource)
• Back in Drupal
  • Ensure your CKAN API base url is added to CKAN Connect settings /admin/config/services/ckan-connect (Configuration > Web services > CKAN Connect). Eg http://YOUR_IP:5000/api/3
  • Create / edit content that has a DVF Ckan field
    • Use the resource URL that you copied above in the Visualisation Url field
    • Select a Visualisation style (eg table) and expand the Data settings section, you should see the fields from your CSV listed. Select a couple of fields (not _id).
    • Save and view your CKAN dataset on your Drupal node!

Checkout some examples for different ways to display your CKAN data.