tableschema-ckan-datastore

tableschema-ckan-datastore-py

Generate and load CKAN DataStore tables based on Table Schema descriptors.

Features

• implements tableschema.Storage interface

Contents

• Getting Started
  • Installation
• Documentation
• API Reference
  • Storage
• Contributing
• Changelog

Getting Started

Installation

The package use semantic versioning. It means that major versions could include breaking changes. It's highly recommended to specify package version range in your setup/requirements file e.g. package>=1.0,<2.0.

pip install tableschema-ckan-datastore

Documentation

When writing data, tableschema-ckan-datastore uses the CKAN API datastore_upsert endpoint with the upsert method. This requires a unique key in the data defined by a Table Schema primary key property. If your data has a primary key, you can use the table.save method:

from tableschema import Table

# Load and save CKAN DataStore record
resource_id = 'bd79c992-40f0-454a-a0ff-887f84a792fb'
base_url = 'https://demo.ckan.org'
dataset_id = 'test-dataset-010203'
api_key = 'my-ckan-user-api-key'

table = Table('data.csv', schema='schema.json')  # data.csv has primary keys
table.save(resource_id,
           storage='ckan_datastore',
           base_url=base_url,
           dataset_id=dataset_id,
           api_key=api_key)

If you need to define the method used to save data to the DataStore, you can create the tableschema.Storage object directly and specify which method parameter to use:

import io
import json
from tabulator import Stream
from tableschema import Storage

# Load and save CKAN DataStore record
resource_id = 'bd79c992-40f0-454a-a0ff-887f84a792fb'
base_url = 'https://demo.ckan.org'
dataset_id = 'test-dataset-010203'
api_key = 'my-ckan-user-api-key'

schema = json.load(io.open('schema.json', encoding='utf-8'))
data = Stream('data.csv', headers=1).open()

storage = Storage.connect('ckan_datastore',
                          base_url=base_url,
                          dataset_id=dataset_id,
                          api_key=api_key)
storage.create(resource_id, schema, force=True))
storage.write(resource_id, data, method='insert')  # specify the datastore_upsert method

API Reference

Storage

Storage(self, base_url, dataset_id=None, api_key=None)

Ckan Datastore storage

Package implements Tabular Storage interface (see full documentation on the link):

Only additional API is documented

Arguments

• base_url (str):
• the base url (and scheme) for the CKAN instance (e.g. http://demo.ckan.org).
• dataset_id (str): id or name of the CKAN dataset we wish to use as the bucket source. If missing, all tables in the DataStore are used.
• api_key (str): either a CKAN user api key or, if in the format env:CKAN_API_KEY_NAME, an env var that defines an api key.

Contributing

The project follows the Open Knowledge International coding standards.

Recommended way to get started is to create and activate a project virtual environment. To install package and development dependencies into active environment:

$ make install

To run tests with linting and coverage:

$ make test

Changelog

Here described only breaking and the most important changes. The full changelog and documentation for all released versions could be found in nicely formatted commit history.

v1.1

• Increase max size of a bucket to 100 resources

v1.0

• Initial driver implementation