Skip to content

opendata-guru/govdata-monitor

Repository files navigation

GovData monitor

image text endpoint svg endpoint data source description
txt svg govdata.de Info box on start page
txt svg govdata.de/web/guest/daten Data counter on data page
txt svg govdata.de/web/guest/sparql-assistent Result of the SPARQL query SELECT (COUNT(?dataset) AS ?datasets) WHERE { ?dataset a dcat:Dataset . }
txt svg govdata.de/web/guest/sparql-assistent Result of the SPARQL query SELECT ?datasets ?distributions (?distributions / ?datasets AS ?averageDistributionsPerDataset) WHERE {{ SELECT (COUNT(?dataset) AS ?datasets) (SUM(?distributionsPerDataset) AS ?distributions) WHERE {{ SELECT ?dataset (COUNT(?distribution) AS ?distributionsPerDataset) WHERE { ?dataset a dcat:Dataset . ?dataset dcat:distribution ?distribution . } GROUP BY ?dataset }}}} LIMIT 100
txt svg ckan.govdata.de/api/3/action/package_list Length of the dataset list on CKAN endpoint
txt svg ckan.govdata.de/catalog.rdf Total items count of pagination info in DCAT-AP.de endpoint and data counter on this page: https://ckan.govdata.de/dataset/
txt svg data.europa.eu/data/datasets?catalog=govdata Data counter for GovData on EU data portal
txt svg data.europa.eu/catalogue-statistics/CurrentState Data counter for GovData on the EU data portal statistic page (updated monthly)
https://ckan.govdata.de/api/3/action/package_search?rows=1&start=0

More endpoints

A monitoring dashboard: https://opendata.guru/govdata/index.html

The OpenAPI description: /openapi.yaml

Count all datasets:

Get a list of all contributors and some more meta data:

Get a list of all datasets:

Get system status information:

Convert RS (Regionalschlüssel) to GEOJSON file:

Parameter length in GET method is limited to some KByte (depend on Browser support). You can use rs as POST parameter to prevent this limitation.

And service endpoints:

(use a cron job service like https://cron-job.org)

Regionalschlüssel:

Ideas

Shapes: Nutzungsbestimmungen und Quellenvermerk

Die hier angebotenen Geodaten stehen über Geodatendienste gemäß der Open Data Datenlizenz Deutschland – Namensnennung – Version 2.0 geldleistungsfrei zum Download und zur Online-Nutzung zur Verfügung. Insbesondere hat jeder Nutzer den Quellenvermerk zu allen Geodaten, Metadaten und Geodatendiensten erkennbar und in optischem Zusammenhang zu platzieren. Veränderungen, Bearbeitungen, neue Gestaltungen oder sonstige Abwandlungen sind mit einem Veränderungshinweis im Quellenvermerk zu versehen. Quellenvermerk und Veränderungshinweis sind wie folgt zu gestalten. Bei der Darstellung auf einer Webseite ist der Quellenvermerk mit der URL "http://www.bkg.bund.de" zu verlinken. © GeoBasis-DE / BKG (Jahr des letzten Datenbezugs) © GeoBasis-DE / BKG (Jahr des letzten Datenbezugs) (Daten verändert)

Organisations

  • start with GovData (or data.europe.eu ?)
  • get a list of all organisations (data providers)
    • the CKAN endpoint for organisations count the datasets
    • classify the data provider per hand (other portal, municipal unit, museum, ...)
    • go to the origial portal / website and count the datasets (if possible)
      • get a list of all organisations ...
      • check for recursion!!!

SPARQL

Go to open data portal SPARQL query page:

Run some queries:

List 10 datasets

PREFIX dcat: <http://www.w3.org/ns/dcat#>
PREFIX odp:  <http://data.europa.eu/euodp/ontologies/ec-odp#>
PREFIX dct: <http://purl.org/dc/terms/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>

SELECT * WHERE { ?d a dcat:Dataset } LIMIT 10

Get a list of all catalogues on data.europa.eu

PREFIX dcat: <http://www.w3.org/ns/dcat#>
PREFIX dct: <http://purl.org/dc/terms/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>

SELECT DISTINCT ?parentCatalog ?catalog ?title ?spatial ?typePublisher ?homepage ?email
WHERE {?catalog ?p ?o.
FILTER (?o=<http://www.w3.org/ns/dcat#Catalog>)
OPTIONAL {?catalog <http://purl.org/dc/terms/title> ?title}
OPTIONAL {?parentCatalog <http://purl.org/dc/terms/hasPart> ?catalog}
OPTIONAL {?catalog <http://purl.org/dc/terms/spatial> ?spatial.}
OPTIONAL {?catalog <http://purl.org/dc/terms/publisher> ?publisher.
OPTIONAL {?publisher <http://xmlns.com/foaf/0.1/homepage> ?homepage.}
OPTIONAL {?publisher <http://xmlns.com/foaf/0.1/mbox> ?email.}
OPTIONAL {?publisher <http://purl.org/dc/terms/type> ?typePublisher.}
}
}
ORDER BY DESC (?spatial)

Publishers by catalogue

PREFIX dcat: <http://www.w3.org/ns/dcat#>
PREFIX odp: <http://data.europa.eu/euodp/ontologies/ec-odp#>
PREFIX dct: <http://purl.org/dc/terms/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>

SELECT DISTINCT ?publisher ?publisherName ?publisherLabel
WHERE {
<http://data.europa.eu/88u/catalogue/govdata> a dcat:Catalog;
dcat:dataset ?datatsetURI.
?datatsetURI dct:publisher ?publisher.
OPTIONAL {?publisher foaf:name ?publisherName}
OPTIONAL {?publisher skos:prefLabel ?publisherLabel}
}
LIMIT 25