CKAN Extensions¶
The TUMI Datahub uses multiple CKAN Extensions. We show the features and how to use each of the extensions.
Which CKAN extension is installed in the TUMI Datahub can be determined by an API call:
https://<datahub-domain>/api/3/action/status_show
Extension "pages"¶
This extension allows you to add additional pages and blog posts to the TUMI Datahub. It's called CKAN's little CMS.
Pages are single static web pages with no relation to a date. Pages can appear in the menu. To create a new page or article, click on the icon for pages (a paper sheet with bent corner).
Blogs are information snippets with a date. They can be used to manually communicate updates of the TUMI datahub, e.g. new datasets. To create a new blog entry click on the icon for blogs (a mall pencil) at the top of any page when logged in.
Configuration details:
- The CKAN Core "About" menu is disabled (ckanext.pages.about_menu=False)
- Enable HTML content in pages (ckanext.pages.allow_html=True)
- We use CKeditor for content editing
The latest blog post is displayed on the start page as widget with a text except.
Pages extensions source code and description
Extensions "datastore" and "xloader"¶
The Datastore and XLoader are for handling tabular data in CKAN.
In CKAN uploaded files are stored in the "CKAN Filestore". If the file contains tabular data (such as in Excel or CSV files), the xLoader grabs the file an pushes it to the internal database Postgres.
The XLoader grabs also tabular file from an URL to upload it to the Datastore.
Datastore¶
When data is stored in the Datastore the
- data preview is much faster
- data can be accessed with the API enabling data queries
- data inserts and updates can be done using the API
When a resource is stored in the Datastore, a button "Data API" is being displayed on the resource page.
Querying data from a resource can be done with the API (e.g. with limit of 5 rows) :
https://<datahub-domain>/api/3/action/datastore_search?resource_id=<resource-id>&limit=5
More information on
datastore
For more information about the Datastore API see CKAN Datastore API Docs.
XLoader¶
The XLoader's job is for uploading data to the Datastore. Either from the internal Filestore or from an URL.
The XLoader works in the background and writes log output to see what was done.
Data managers can see the XLoader log in Datasets -> Resources -> Manage -> Datastore. The log entries show the data resource being uploaded to the database.
Full docs about the XLoader is on Github:
ckanext-xloader
Advanced XLoader configuration¶
Administrator can use additional configuration to customize the xLoader using environment variables to the CKAN configuration.
Extension "matomo_dsgvo"¶
In Tumi Open Data Portals we use ckanext-matomo-dsgvo extension to enable analytics with data privacy.
This extension puts the Matomo tracking script into the web pages, as well as a matching cookie-banner.
These varables are already set in the ckan config file, but you may need to adjust them:
The variables are:
matomo_dsgvo.site_id=
matomo_dsgvo.url=
matomo_dsgvo.datenschutzseite=/pages/privacy
This guide explains, how to customize these environment variables.
To access and set up Matomo visit https://analytics.CITYID.tumidata.org There follow the setup guide and then you can check your Matomo data there.
Extension "showcase"¶
This extension allows to create showcases for apps or websites.
A showcase (aka App) helps to tell the user how to use a dataset. For example this could be a website which can be feeded with this dataset's data or an Excel spreadsheet to easily process the Dataset's resources.
Only CKAN users, who are sysadmins or have the role of a showcase admin can create or add new showcases.
Extension "dcat"¶
DCAT is an RDF vocabulary designed to facilitate interoperability between data catalogs published on the Web (see definition at W3C).
Sounds complicated? In easy words: DCAT is a metatdata standard to help understand data better and to prepare data processing by algorithms.
A good start to understand DCAT is the webinar series DCAT and DCAT-AP training: General introduction
To see the DCAT XML representation of the entire open data portal, past this url to the browser (in Firefox, show the page source afterwards):
https://<datahub-domain>/catalog.rdf
More information about DCAT in CKAN:
The DCAT extension for CKAN
General DCAT specification by W3C
Extension "spatial"¶
This extension adds a bunch of spacial capabilities to the CKAN open data portal:
- Some spatial metadata field for datasets
- A map display for datasets which shows the geographical extent of the dataset
- Map based search for datasets
When starting with spatial data for datasets, most of the data should have these spatial metadata because otherwise the map based search will not find any results.
More information in:
The Github repo of the Spatial Extension
CKAN Spatial Extension Docs
Extension "scheming"¶
This extension allows you to configure the ckan schema for datasets, organizations, groups and resources.
In other words, you can add additional metadata fields.
Extension "fluent"¶
This extension allows you to use multilingual fields for datasets, resources, organizations and groups.
For example you can write the dataset description for different languages.
Then website visitors will see the respective description for this dataset depending on their language settings.
Extensions "*view"¶
These extensions allow to display resources on the ckan site without downloading it.
recline_view (on website: "Data Explorer") for tabular data, like csv, xls and xlsx
image_view to display images, possible formats are png, jpeg, jpg and gif.
text_view to display xml, json and txt files.
geo_view to display geospatial files.
pdf_view to display pdfs.
For recline_view to work properly, the tabular data has to be uploaded to the datastore.
The extension resource_proxy is necessary to allow view plugins to work on files located in a different domain.
More information about:
Views in general
Geoview
Pdfview
Extension "check-link"¶
This extension allows to check the resources on the ckan site for broken links.
Click on the tab "Check Links" in the user dashboard.
The button "Check" inspects all resource links and opens the result in JSON format in a new tab.
The button "Results" shows the results of the last inspection.
Only CKAN users, who are sysadmins can use this function.