datacrunch

DataCrunch Python SDK

The official DataCrunch.io Python SDK.

The SDK's documentation is available on ReadTheDocs

DataCrunch's Public API documentation is available here.

Getting Started - Using the SDK:

• Install via pip:

  pip3 install datacrunch
  

• Generate your client credentials - instructions in the public API docs.

• Add the client secret to an environment variable (don't want it to be hardcoded):

  Linux (bash):

  export DATACRUNCH_CLIENT_SECRET=Z4CZq02rdwdB7ISV0k4Z2gtwAFKiyvr2U1l0KDIeYi
  

  Other platforms: https://en.wikipedia.org/wiki/Environment_variable

• Example for creating a new instance:

  import os
  from datacrunch import DataCrunchClient
  
  # Get client secret from environment variable
  CLIENT_SECRET = os.environ['DATACRUNCH_CLIENT_SECRET']
  CLIENT_ID = 'Ibk5bdxV64lKAWOqYnvSi'
  
  # Create datcrunch client
  datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET)
  
  # Get all SSH keys
  ssh_keys = datacrunch.ssh_keys.get()
  ssh_keys = list(map(lambda key: key.id, ssh_keys))
  
  # Create a new instance
  instance = datacrunch.instances.create(instance_type='1V100.6V',
                                        image='fastai',
                                        ssh_key_ids=ssh_keys,
                                        hostname='example',
                                        description='example instance')
  
  # Delete instance
  datacrunch.instances.action(instance.id, datacrunch.constants.instance_actions.DELETE)
  

  More examples can be found in the /examples folder or in the documentation.

Development

Setting up the local development environment

• Clone the repository:

  git clone
  

• Create local virtual environment:

  python3 -m venv datacrunch_env && source ./datacrunch_env/bin/activate
  

  or if using fish shell:

  python3 -m venv datacrunch_env && source ./datacrunch_env/bin/activate.fish
  

• Install Dependencies:

  pip3 install -e .[test]
  pip3 install -U pytest
  

Running Tests

We use pytest for testing.

• To execute all tests

  pytest
  

• To execute a single test file

  pytest ./tests/unit_tests/test_file.py
  

Local Manual Testing

Create this file in the root directory of the project:

from datacrunch.datacrunch import DataCrunchClient

CLIENT_SECRET = 'secret'
CLIENT_ID = 'Ibk5bdxV64lKAWOqYnvSi'

# Create datcrunch client
datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET, base_url='http://localhost:3001/v1')

Generating the documentation

If added a new service, create a documentation template under api/services for that service.

cd docs
make html

Style Guide

Use autopep8 for auto code formatting:

# Install
pip3 install autopep8

# Apply to an entire directory
autopep8 directory_name --recursive --in-place --pep8-passes 2000 --verbose

# Or a single file
autopep8 file.py --in-place

Contact

You can contact us here, or send a message / open an issue in the repo.