Socket
Socket
Sign inDemoInstall

cloudstorage

Package Overview
Dependencies
24
Maintainers
1
Alerts
File Explorer

Install Socket

Protect your apps from supply chain attacks

Install

cloudstorage

Unified cloud storage API for storage services.

    0.11.0

Maintainers
1

Readme

=============
Cloud Storage
=============



`Cloud Storage`_ is a Python +3.5 package which creates a unified API for the
cloud storage services: Amazon Simple Storage Service (S3),
Microsoft Azure Storage, Minio Cloud Storage, Rackspace Cloud Files,
Google Cloud Storage, and the Local File System.

Cloud Storage is inspired by `Apache Libcloud <https://libcloud.apache.org/>`_.
Advantages to Apache Libcloud Storage are:

* Full Python 3 support.
* Generate temporary signed URLs for downloading and uploading files.
* Support for request and response headers like Content-Disposition.
* Pythonic! Iterate through all blobs in containers and all containers in
  storage using respective objects.

Usage
=====

.. code-block:: python

    >>> from cloudstorage.drivers.amazon import S3Driver
    >>> storage = S3Driver(key='<my-aws-access-key-id>', secret='<my-aws-secret-access-key>')

    >>> container = storage.create_container('avatars')
    >>> container.cdn_url
    'https://avatars.s3.amazonaws.com/'

    >>> avatar_blob = container.upload_blob('/path/my-avatar.png')
    >>> avatar_blob.cdn_url
    'https://s3.amazonaws.com/avatars/my-avatar.png'

    >>> avatar_blob.generate_download_url(expires=3600)
    'https://avatars.s3.amazonaws.com/my-avatar.png?'
    'AWSAccessKeyId=<my-aws-access-key-id>'
    '&Signature=<generated-signature>'
    '&Expires=1491849102'

    >>> container.generate_upload_url('user-1-avatar.png', expires=3600)
    {
        'url': 'https://avatars.s3.amazonaws.com/',
        'fields': {
            'key': 'user-1-avatar.png',
            'AWSAccessKeyId': '<my-aws-access-key-id>',
            'policy': '<generated-policy>',
            'signature': '<generated-signature>'
        }
    }

Supported Services
==================

* `Amazon S3`_
* `Google Cloud Storage`_
* `Microsoft Azure Storage`_
* `Minio Cloud Storage`_
* `Rackspace CloudFiles`_
* Local File System


Installation
============

To install Cloud Storage:

.. code-block:: bash

    pip install cloudstorage

Also install the storage driver(s) you will be using:

.. code-block:: bash

    pip install cloudstorage[amazon]
    pip install cloudstorage[google]
    pip install cloudstorage[local]
    pip install cloudstorage[microsoft]
    pip install cloudstorage[minio]
    pip install cloudstorage[rackspace]

.. _`Amazon S3`: https://aws.amazon.com/s3/
.. _`Blackblaze B2 Cloud Storage`: https://www.backblaze.com/b2/Cloud-Storage.html
.. _`Google Cloud Storage`: https://cloud.google.com/storage/
.. _`Microsoft Azure Storage`: https://azure.microsoft.com/services/storage/
.. _`Minio Cloud Storage`: https://www.minio.io/
.. _`Rackspace CloudFiles`: https://www.rackspace.com/cloud/files
.. _`Cloud Storage`: https://github.com/scottwernervt/cloudstorage/

.. :changelog:

Changelog
---------

1.0.0 (Unreleased)
+++++++++++++++++++

Features

* Start following (`Semantic Versioning <https://semver.org/>`_).


0.11.0 (2021-01-15)
+++++++++++++++++++

Features

* Atomic local file write (`#68 <https://github.com/scottwernervt/cloudstorage/pull/68>`_). Thanks @habibutsu.
* New DigitalOcean Spaces Driver (`#67 <https://github.com/scottwernervt/cloudstorage/pull/67>`_). Thanks @RangelReale.

Bugs

* Convert ``bytes`` to ``str`` before saving the json file for Windows Xattr simulator (`#66 <https://github.com/scottwernervt/cloudstorage/pull/66>`_). Thanks @RangelReale!
* Handle ``pathlib.Path`` properly for ``Blob.download`` download and ``Container.upload_blob`` (`#65 <https://github.com/scottwernervt/cloudstorage/pull/65>`_). Thanks @sibowsb!
* Fix Bob ``NotFoundError`` message for AWS driver (`#64 <https://github.com/scottwernervt/cloudstorage/pull/64>`_). Thanks @sibowsb!

0.10.1 (2020-04-20)
+++++++++++++++++++

Features

* Add support to mimic ``xattr`` in Windows by storing data in ``.<filename>.xattr`` (`#56 <https://github.com/scottwernervt/cloudstorage/pull/56>`_). Thanks @RangelReale.
* Add support for ``python 3.8``.

Bugs

* Freeze package ``azure`` to version `4.0.0` (`#58 <https://github.com/scottwernervt/cloudstorage/issues/58>`_).
* Drop support for ``python 3.4`` due to ``PyYAML`` requirement ``!=3.4.*``.

0.10.0 (2019-08-10)
+++++++++++++++++++

Bugs

* Azure ``get_blob`` raises ``TypeError`` when Content MD5 missing in remote (`#47 <https://github.com/scottwernervt/cloudstorage/issues/47>`_). Thanks @matt-carr.
* ``AzureStorageDriver`` reads stream twice without rewinding (`#44 <https://github.com/scottwernervt/cloudstorage/issues/44>`_). Thanks @matt-carr.
* Update package dependencies to fix tests (`#43 <https://github.com/scottwernervt/cloudstorage/issues/43>`_).

Changes from 0.9.0:

* Azure driver will issue a warning if Content MD5 is not populated.
* Google driver tests will not run if credentials file is not present.

0.9.0 (2018-11-29)
++++++++++++++++++

Features

* Driver authentication can be verified using ``DriverName.validate_credentials()`` (`#34 <https://github.com/scottwernervt/cloudstorage/issues/34>`_).

Changes from 0.8.0:

* Initializing ``GoogleStorageDriver`` with an invalid credentials file will
  raise ``CredentialsError`` exception instead of ``CloudStorageError``.

0.8.0 (2018-11-06)
++++++++++++++++++

Features

* ``Blob`` and ``Container``'s ``meta_data`` is now a case insensitive dictionary.
* Add new driver for Minio Cloud Storage (`#25 <https://github.com/scottwernervt/cloudstorage/issues/25>`_).
  Install driver requirements with: ``pip install cloudstorage[minio]``.

Other

* Move to ``src`` folder structure for package.

0.7.0 (2018-10-03)
++++++++++++++++++

Features

* ``Cache-Control`` supported for Amazon, Google, Local, and Microsoft (`#11 <https://github.com/scottwernervt/cloudstorage/issues/11>`_).
* Each driver's package dependencies are now optional (`#4 <https://github.com/scottwernervt/cloudstorage/issues/4>`_).

Other

* Remove rackspace package dependency ``rfc6266_parser``.
* Add ``flake8`` linting and ``sphinx`` doc building to tox and travis.

0.6 (2018-07-24)
++++++++++++++++

* Copy metadata from ``setup.py`` to ``setup.cfg``
* Add rate limit timeout when calling google cloud storage backend during tests.
* Catch ``UnicodeDecodeError`` when decoding local file attribute values.
* Upgrade dependencies and include ``requirements.txt`` and ``dev-requirements.txt``.

0.5 (2018-02-26)
++++++++++++++++

* Update rackspacesdk to 0.7.5 and fix broken API calls (`#14 <https://github.com/scottwernervt/cloudstorage/issues/14>`_).

0.4 (2017-08-29)
++++++++++++++++

* Implement Microsoft Azure Storage driver (`#1 <https://github.com/scottwernervt/cloudstorage/issues/1>`_).
* Google upload_blob is failing for binary stream (`#7 <https://github.com/scottwernervt/cloudstorage/issues/7>`_ and `#8 <https://github.com/scottwernervt/cloudstorage/issues/8>`_).
* Fixed type annotations using mypy.
* Formatted code using flake8 recommendations.

0.3 (2017-05-24)
++++++++++++++++

* Fixes `#6 <https://github.com/scottwernervt/cloudstorage/issues/6>`_: Add kwargs to each driver's init method.

0.2 (2017-04-21)
++++++++++++++++

* Add pip cache to travis yml file to speed up tests.
* Set wheel python-tag to py3 only
* Set tox to pass all env variables to py.test
* Add travis repo encrypted env variables for running tests.

0.1 (2017-04-20)
++++++++++++++++

* First release.

Keywords

FAQs


Did you know?

Socket installs a GitHub app to automatically flag issues on every pull request and report the health of your dependencies. Find out what is inside your node modules and prevent malicious activity before you update the dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc