thumbor-aws
This is a project to provide modern thumbor>7.0.0 AWS Extensions.
⚙️ Installation
pip install thumbor-aws
🎯 Features
- Asynchronous non-blocking AWS S3 support
- Conforms with thumbor 7 new storage and results storage specs
- Python 3 compliant
- S3 Image Loader - Load source images from S3
- S3 Storage - Retrieve and store source images, detector data and security keys
- S3 Result Storage - Retrieve and store resulting images
- Compatibility mode for users of tc_aws: currently supported loader, storage and result storage
Usage
Configuring thumbor
Configure your thumbor.conf
file to point to thumbor_aws
:
## The loader thumbor should use to find source images.
## This must be the full name of a python module (python must be able to import it)
LOADER = "thumbor.loaders.http_loader"
## The file storage thumbor should use to store original images.
## This must be the full name of a python module (python must be able to import it)
STORAGE = 'thumbor_aws.storage'
## The result storage thumbor should use to store generated images.
## This must be the full name of a python module (python must be able to import it)
RESULT_STORAGE = 'thumbor_aws.result_storage'
You should use only the extensions required by your use case.
There's no dependency between them.
Configuration
thumbor-aws allows you to configure each extension independently:
General
Some S3 providers fail to return a valid location header when uploading a new object. For that scenario, thumbor-aws
allows users to set the location template to be used.
## Default location to use if S3 does not return location header.
## Can use {bucket_name} var.
## Defaults to: 'https://{bucket_name}.s3.amazonaws.com'
AWS_DEFAULT_LOCATION = "https://{bucket_name}.s3.amazonaws.com"
Loader
thumbor-aws loader offer several configuration options:
Storage
Below you can see the result of running thumbor's config generation after importing thumbor-aws:
################################# AWS Storage ##################################
## Region where thumbor's objects are going to be stored.
## Defaults to: 'us-east-1'
#AWS_STORAGE_REGION_NAME = 'us-east-1'
## S3 Bucket where thumbor's objects are going to be stored.
## Defaults to: 'thumbor'
#AWS_STORAGE_BUCKET_NAME = 'thumbor'
## Secret access key for S3 to allow thumbor to store objects there.
## Defaults to: None
#AWS_STORAGE_S3_SECRET_ACCESS_KEY = None
## Access key ID for S3 to allow thumbor to store objects there.
## Defaults to: None
#AWS_STORAGE_S3_ACCESS_KEY_ID = None
## Endpoint URL for S3 API. Very useful for testing.
## Defaults to: None
#AWS_STORAGE_S3_ENDPOINT_URL = None
## Storage prefix path.
## Defaults to: '/st'
#AWS_STORAGE_ROOT_PATH = '/st'
## Storage ACL for files written in bucket
## Defaults to: 'public-read'
#AWS_STORAGE_S3_ACL = 'public-read'
## Default location to use if S3 does not return location header. Can use
## {bucket_name} var.
## Defaults to: 'https://{bucket_name}.s3.amazonaws.com'
#AWS_DEFAULT_LOCATION = 'https://{bucket_name}.s3.amazonaws.com'
################################################################################
Result Storage
Below you can see the result of running thumbor's config generation after importing thumbor-aws:
############################## AWS Result Storage ##############################
## Region where thumbor's objects are going to be stored.
## Defaults to: 'us-east-1'
#AWS_RESULT_STORAGE_REGION_NAME = 'us-east-1'
## S3 Bucket where thumbor's objects are going to be stored.
## Defaults to: 'thumbor'
#AWS_RESULT_STORAGE_BUCKET_NAME = 'thumbor'
## Secret access key for S3 to allow thumbor to store objects there.
## Defaults to: None
#AWS_RESULT_STORAGE_S3_SECRET_ACCESS_KEY = None
## Access key ID for S3 to allow thumbor to store objects there.
## Defaults to: None
#AWS_RESULT_STORAGE_S3_ACCESS_KEY_ID = None
## Endpoint URL for S3 API. Very useful for testing.
## Defaults to: None
#AWS_RESULT_STORAGE_S3_ENDPOINT_URL = None
## Result Storage prefix path.
## Defaults to: '/rs'
#AWS_RESULT_STORAGE_ROOT_PATH = '/rs'
## ACL to use for storing items in S3.
## Defaults to: None
#AWS_RESULT_STORAGE_S3_ACL = None
################################################################################
Configuring thumbor in compatibility mode with tc_aws
If you are a tc_aws user, thumbor-aws has a compatibility mode where you can use the same configuration you are already used to:
THUMBOR_AWS_RUN_IN_COMPATIBILITY_MODE = True
TC_AWS_REGION = 'us-east-1'
TC_AWS_MAX_RETRY = 0
TC_AWS_LOADER_BUCKET = 'my-bucket'
TC_AWS_LOADER_ROOT_PATH = 'source-files'
TC_AWS_STORAGE_BUCKET = 'my-bucket'
TC_AWS_STORAGE_ROOT_PATH = 'source-files'
TC_AWS_STORAGE_SSE = False
TC_AWS_STORAGE_RRS = False
TC_AWS_RESULT_STORAGE_BUCKET = 'my-bucket'
TC_AWS_RESULT_STORAGE_ROOT_PATH = 'result-storage'
TC_AWS_STORE_METADATA = False
Please notice the addition of THUMBOR_AWS_RUN_IN_COMPATIBILITY_MODE = True
to tell thumbor_aws
you want compatibility with tc_aws
.
If you have any issues with this
Caveats
- thumbor-aws does not create buckets for you. If they don't exist you are getting errors.
You can easily create a bucket using aws-cli with:
$ aws s3api create-bucket --bucket <bucket name> --region <your region>
Or through your AWS Console UI.
Troubles?
If you experience any troubles, try running:
thumbor-doctor
If you still need help, please raise an issue.
👀 Thumbor
thumbor-aws stands on the shoulders of thumbor! If you are not familiar with thumbor, please check the docs or you can see a demo at http://thumborize.me/
👍 Contribute
thumbor-aws is an open-source project with many contributors. Join them
contributing code or
contributing documentation.
Join the chat at https://gitter.im/thumbor/thumbor
License
MIT License
Copyright (c) 2021 thumbor-aws (by @heynemann)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.