ckanext-csvwmapandtransform

ckanext-csvwmapandtransform

Extension automatically generating csvw metadata for uploaded textual tabular data. It uploads the data of the first table documented into a datastore for the source csv file.

Requirements

Needs a running instance of the MapToMethod Application and RDFConverter Application Point at it through env variables. Also needed is a Api Token for an account with the right privaledges to make the background job work on private datasets and ressources.

CSVWMAPANDTRANSFORM_TOKEN=${CKAN_API_TOKEN}
MAPTOMETHOD_CONTAINER_NAME="ckan_maptomethod"
MAPTOMETHOD_APP_PORT=5002
# must be reachable from outside container net or iframe wil not work 
CKAN_MAPTOMETHOD_URL=http://<CKAN_HOST>:${MAPTOMETHOD_APP_PORT}
RDFCONVERTER_CONTAINER_NAME="ckan_rdfconverter"
RDFCONVERTER_APP_PORT=5003
CKAN_RDFCONVERTER_URL=http://${RDFCONVERTER_CONTAINER_NAME}:${RDFCONVERTER_APP_PORT}
CSVWMAPANDTRANSFORM_SQLALCHEMY_URL=postgresql://<ckandbuser>:<ckandbpassword>@<db>/ckandb
PARSER_PORT=3001
MAPPER_PORT=4000
CONVERTER_PORT=5000

You can set the default formats to run trusformation on by setting the env variable CSVWMAPANDTRANSFORM_FORMATS for example

CSVWMAPANDTRANSFORM_FORMATS="json-ld turtle n3 nt hext trig longturtle xml"

else it will react to the following formats: "json json-ld turtle n3 nt hext trig longturtle xml"

CSVWMAPANDTRANSFORM_FORMATS="json json-ld turtle n3 nt hext trig longturtle xml"

Compatibility with core CKAN versions:

CKAN version	Compatible?
2.9 and arlier	not tested
2.10	yes
2.11	yes

• "yes"
• "not tested" - I can't think of a reason why it wouldn't work
• "not yet" - there is an intention to get it working
• "no"

Installation

To install the extension:

• Activate your CKAN virtual environment, for example:

. /usr/lib/ckan/default/bin/activate

• Use pip to install package

pip install ckanext-csvwmapandtransform

• Add csvwmapandtransform to the ckan.plugins setting in your CKAN    config file (by default the config file is located at    /etc/ckan/default/ckan.ini).

• Restart CKAN. For example, if you've deployed CKAN with Apache on Ubuntu:

sudo service apache2 reload

Developer installation

To install ckanext-csvtocsvw for development, activate your CKAN virtualenv and do:

git clone https://github.com/Mat-O-Lab/ckanext-csvtocsvw.git
cd ckanext-csvtocsvw
python setup.py develop
pip install -r dev-requirements.txt

Tests

To run the tests, do:

pytest --ckan-ini=test.ini

License

AGPL

Acknowledgments

The authors would like to thank the Federal Government and the Heads of Government of the Länder for their funding and support within the framework of the Platform Material Digital consortium. Funded by the German Federal Ministry of Education and Research (BMBF) through the MaterialDigital Call in Project KupferDigital - project id 13XP5119.