azure-utils

Azure Utils

Azure Utilities for python

Documentation

Technical documentation can be found here.

Setup

Make sure you have Python 3.6.x (or higher) installed on your system. You can download it here.

Recommended (but Optional) -> Expand this section to setup and activate a virtual environment.

• Install (or upgrade) virtualenv:

python3 -m pip install --upgrade virtualenv

• Create your virtualenv named venv:

python3 -m virtualenv venv

• Activate your virtual environment
  • On Unix (Mac or Linux):

  source venv/bin/activate
  

  • On Windows:

  venv\scripts\activate
  

pip install azure_utils

Basic Usage

import os, sys
from azure_utils.blob_storage import AZContainer

# Get the working directory of this file
wd=os.path.abspath(os.path.dirname(sys.argv[0]))

# Create a container object
az = AZContainer(
    account_url="https://<account_name>.blob.core.windows.net",
    account_key="<account_key>",
    container_name="<container_name>"
)

# Get a list of all files in the container and print the first 5
files = az.list_files(remote_folderpath='/')
print(files[:5])

# Upload a file to the container
az.upload_file(
    remote_filepath='/test.csv',
    local_filepath=f'{wd}/test_data/upload/test.csv',
    overwrite=True
)

# Download the file from the container
az.download_file(
    remote_filepath='/test.csv',
    local_filepath=f'{wd}/test_data/download/test.csv',
    overwrite=True,
    smart_sync=True
)

# Delete the file from the container
az.delete_file(
    remote_filepath='/test.csv'
)

# Upload a folder to the container
az.sync_to_remote(
    remote_folderpath='/',
    local_folderpath=f'{wd}/test_data/upload/',
    overwrite=True,
)

# Download the folder from the container
az.sync_to_local(
    remote_folderpath='/',
    local_folderpath=f'{wd}/test_data/download/',
    overwrite=True,
    smart_sync=True,
)

# Delete all files in the container
az.delete_folder(
    remote_folderpath='/',
)

# Delete all local metadata
az.clear_local_meta(f'{wd}/test_data/')

Contributors:

• Alice Zhao
• Connor Makowski