Bucket Client is a ruby gem that allows programmers to interact with popular Blob Storage cloud services. This intends to act as a layer of abstraction, much like ORM is to databases.
With this, you may easily change the blob storage provider or even defer them.
The supported cloud storage include:
Add this line to your application's Gemfile:
gem 'bucket_client'
And then execute:
$ bundle
Or install it yourself as:
$ gem install bucket_client
To begin using BucketClient, you have to create a Client object based on the cloud service you want to use.
It is advised to use the LocalClient for development and test, as it uses the local disk to simulate how
online blob storage will work, except for set_get_cors and set_read_policy functions. This ensure fast
tests and development without polluting the actual bucket.
The creation of client use the generate class-level (or rather, module-level) method of the BucketClient module.
Local Client uses a local path to simulate storage of blob. The path variable is the relative path from
terminal that it uses to simulate storage. Please be careful as it may delete files within that folder.
To create a local client:
require "bucket_client"
client = BucketClient::generate type: :local, path: "./public/sample-bucket"
client # => the client used to perform CRUD on bucket and blobs
AWS Client requires 3 values, the access id, access key, which is the secret, and the region.
To create a AWS S3 client:
require "bucket_client"
client = BucketClient::generate type: :aws, id: ENV["AWS_ID"], secret: ENV["AWS_SECRET"], region: ENV["AWS_REGION"]
client # => the client used to perform CRUD on bucket and blobs
Digital Ocean spaces requires 3 values, the access id, access key, which is the secret, and the region.
To create a Digital Ocean client:
require "bucket_client"
client = BucketClient::generate type: :spaces, id: ENV["AWS_ID"], secret: ENV["AWS_SECRET"], region: ENV["AWS_REGION"]
client #=> the client used to perform CRUD on bucket and blobs
Azure Blob Storage require 2 values, the account name and the key, which is the secret.
To create a Azure client:
require "bucket_client"
client = BucketClient::generate type: :azure, id: ENV["AZURE_ACC_NAME"], secret: ENV["AZURE_KEY"]
client #=> the client used to perform CRUD on bucket and blobs
GCP Cloud Storage require 2 value, the project_id and the secret
The secret can be passed in via 2 methods,
To create a GCP client using the Hash object (assume the JSON value is stored as environment variable):
require "json"
require "bucket_client"
secret = JSON.parse(ENV["GOOGLE_KEY"])
client = BucketClient::generate type: :gcp, id: ENV["GOOGLE_ID"], secret: secret
client #=> the client used to perform CRUD on bucket and blobs
To create a GCP client using the path to json secret:
require "bucket_client"
client = BucketClient::generate type: :gcp, id: ENV["GOOGLE_ID"], secret: "path/to/secret.json"
client #=> the client used to perform CRUD on bucket and blobs
OperationResult is the object you obtain from the normal operations. It contains details of the operation where you can check:
| Property | Description |
|---|---|
| success | whether the operation was successful. Boolean value |
| code | the status code of the operation |
| message | the message of the operation. Error messages can be checked here |
| value | the usable value of the operation. May be url or binary |
If you rather immediately obtain the value, and raise error when it is unsuccessful, you may use the ! version of the method.
# Using OperationResult Object
result = bucket.create_blob binary, "path/to/bucket/file.bin"
if result.success
p result.code #=> prints the status code
p result.message #=> prints the success message, if any
p result.value # => prints the URI obtain from "create_blob" method, when successful
else
p result.code #=> check what HTTP error code is obtained
p result.message #=> check error message
end
# Or use ! method to immediate capture the method
begin
result = bucket.create_blob! binary, "path/to/bucket/file.bin"
p result #=> prints the URI obtained from "create_blob" method, when successful
rescue StandardError => e
p e.message #=> prints the error message. This will include the status code
end
The client object obtain via the generate method can be used to perform Bucket CRUD actions. It works across
all platforms.
bang methods (methods that end with !) do not return OperationResult. Instead, they raise error if they fail.
key:string => booleanChecks whether the bucket of a certain key exist. Raises exception when the HTTP request underneath fails
# client from above
exist = client.exist_bucket "sample-bucket"
exist #=> true if exist, false if it does not exist
key:string => OperationResult<Bucket>Creates a bucket using the provided key. Fails if bucket already exist.
value of OperationResult if successful is the Bucket object that has been created.
# client from above
result = client.create_bucket "sample-bucket"
if result.success
bucket = result.value
bucket #=> obtains the bucket
else
p result.message #=> prints the error message
p result.code #=> prints the status code
end
key:string => BucketCreates a bucket using the provided key. Fails if bucket already exist.
Raises exception if fail, returns Bucket object that has been created if it succeeds
# client from above
result = client.create_bucket "sample-bucket"
result #=> obtains bucket
key:string => OperationResult<nil>Deletes the bucket provided in the key. Fails if bucket does not exist.
To prevent that behaviour, use delete_bucket_if_exist to not fail even if
bucket does not exist.
Does not return anything on success. value will always return nil
# client from above
result = client.delete_bucket "sample-bucket"
result.success #=> whether the bucket has been successfully deleted
result.message #=> Error message or success message
result.code #=> status code of the operation
key:string => nilDeletes the bucket provided in the key. Fails if bucket does not exist.
To prevent that behaviour, use delete_bucket_if_exist! to not fail even if
bucket does not exist.
Raises exception if fail. Returns nil
# client from above
client.delete_bucket! "sample-bucket" #=> nil
key:string => OperationResult<nil>Deletes the bucket provided in the key. Will succeed even if bucket does not exist.
Does not return anything on success. value will always return nil
# client from above
result = client.delete_bucket_if_exist "sample-bucket"
result.success #=> whether the bucket has been successfully deleted
result.message #=> Error message or success message
result.code #=> status code of the operation
key:string => nilDeletes the bucket provided in the key. Will succeed even if bucket does not exist.
Raises exception if the operation fails
# client from above
client.delete_bucket_if_exist! "sample-bucket" #=> nil
key:string => OperationResult<Bucket>Creates the bucket provided in key if it does not exist.
This method will succeed and return the bucket even if the bucket exist, unlike create_bucket
value of the OperationResult is the Bucket object if the operation succeeds
# client from above
result = client.put_bucket "sample-bucket"
if result.success
bucket = result.value
bucket #=> obtains the bucket
else
p result.message #=> prints the error message
p result.code #=> prints the status code
end
key:string => BucketCreates the bucket provided in key if it does not exist.
This method will succeed and return the bucket even if the bucket exist, unlike create_bucket
Returns the Bucket object
Raises exception if the operation fails
# client from above
bucket = client.put_bucket! "sample-bucket"
bucket #=> obtains the bucket that has been creted
key:string, access:symbol => OperationResult<nil>Sets the read policy of the bucket. This does not work for LocalBucket as LocalBucket does not have
concept of "access".
Only two values are accepted: :public and :private.
:public allows everyone with access to the link to read the blobs within the bucket
:private only allows people with authorization (with secret key) to read the blob within the bucket
# client from above
result = client.set_read_policy "sample-bucket", :public
result.success #=> whether the bucket has been made public
result.message #=> Error message or success message
result.code #=> status code of the operation
key:string, access:symbol => nilSets the read policy of the bucket. This does not work for LocalBucket as LocalBucket does not have
concept of "access".
Raises exception if the operation fails.
Only two values are accepted: :public and :private.
:public allows everyone with access to the link to read the blobs within the bucket
:private only allows people with authorization (with secret key) to read the blob within the bucket
# client from above
client.set_read_policy! "sample-bucket", :public #=> nil
key:string, cors:array<string> => OperationResult<nil>Sets the GET CORS of the bucket. This is limits the Cross Origin Resource Sharing to the domains
within the cors array you input. To allow all origin, please use ["*"] as cors value.
This does not work for LocalBucket as it does not have concept of cors.
This is one-level higher for AzureBucket, where it modifies the whole accounts' CORS, not just the bucket.
# client from above
result = client.set_get_cors "sample-bucket", ["*"]
result.success #=> whether it has succeeded allowing all origin to read
result.message #=> Error message or success message
result.code #=> status code of the operation
key:string, cors:array<string> => nilSets the GET CORS of the bucket. This is limits the Cross Origin Resource Sharing to the domains
within the cors array you input. To allow all origin, please use ["*"] as cors value.
This does not work for LocalBucket as it does not have concept of cors.
This is one-level higher for AzureBucket, where it modifies the whole accounts' CORS, not just the bucket.
Raises exception if the operation fails
#client from above
client.set_get_cors! "sample-bucket", ["*"] #=> nil
key:string => BucketObtains the Bucket instance with the key.
The bucket instance can be used to perform CRUD for blobs within the bucket.
This method will raise exception if the bucket does not exist. To improve speed as you are sure that the bucket
already exist, please use the bang version, get_bucket!, where it will not do a look up.
#client from above
bucket = get_bucket "sample-bucket"
bucket #=> bucket instance obtained.
key:string => BucketObtains the Bucket instance with the key.
The bucket instance can be used to perform CRUD for blobs within the bucket.
This method will not do a look up, so you instance's blob CRUD operation may fail if you did not verify the existence of this bucket. This performs faster than the non-bang version as it does not spend operation to check existence of the bucket, making the assumption that it exist.
#client from above
bucket = get_bucket! "sample-bucket"
bucket #=> bucket instance obtained.
The client object can perform Blob CRUD if it has access to the full URI or URL of the blob.
Bang methods (methods that end with !) do not return OperationResult. Instead, they raise error if they fail.
uri:string => OperationResult<array<byte>>Obtains the binary of the blob via the URI of the blob.
value of the OperationResult is the byte array of the binary if the operation succeeds
# client from above
result = client.get_blob "https://host.com/bucket/blob.bin"
if result.success #=> whether the obtaining of the blob succeeded
binary = result.value #=> obtain the binary value
IO.binwrite "blob.bin", binary #=> writes it to disk
else
p result.message #=> Error message or success message
p result.code #=> status code of the operation
end
uri:string => <array<byte>>Obtains the binary of the blob via the URI of the blob
Raises exception if it fails
binary = client.get_blob! "https://host.com/bucket/blob.bin"
IO.binwrite "blob.bin", binary
uri:string => booleanChecks whether the blob exist
exist = client.exist_blob "https://host.com/bucket/blob.bin"
exist #=> true if blob exist, false if it doesn't
payload:array<byte>, uri:string => Operation<string>Updates a blob with new payload in byte array
value of the OperationResult will return URI of the blob if success
Fails if blob with the URI doesn't exist
img = IO.binread "pic.png"
uri = "https://host.com/folder/pic.png"
result = client.update_blob img, uri
result.success #=> Whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> Uri of update blob
payload:array<byte>, uri:string => stringUpdates a blob with new payload in byte array Fails if blob doesnt exist
Raises exception if operation fails
img = IO.binread "pic.png"
uri = "https://host.com/folder/pic.png"
result = client.update_blob! img, uri
result #=> URI of update blob
payload:array<byte>, uri:string => OperationResult<string>Creates the blob with the payload if it does not exist, updates the blob with the new payload if it exist
value of the OperationResult will return URI of the blob if success
img = IO.binread "pic.png"
uri = "https://host.com/folder/pic.png"
result = client.put_blob img, uri
result.success #=> Whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> Uri of blob
payload:array<byte>, uri:string => stringCreates the blob with the payload if it does not exist, updates the blob with the new payload if it exist
Raises exception if the operation fails
img = IO.binread "pic.png"
uri = "https://host.com/folder/pic.png"
result = client.put_blob! img, uri
result #=> returns URI of updated blob
uri:string => OperationResult<nil>Deletes the blob in the provided URI
Fails if the blob does not exist. Use delete_blob_if_exist if you do not want this behaviour
value of OperationResult is always nil
uri = "https://host.com/folder/pic.png"
result = client.delete_blob uri
result.success #=> Whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> nil
uri:string => nilDeletes the blob in the provided URI Fails if the blob does not exist. Use delete_blob_if_exist if you do not want this behaviour
Raises exception if the operation fails
uri = "https://host.com/folder/pic.png"
client.delete_blob! uri
uri:string => OperationResult<nil>Deletes the blob if it exist, else does nothing
value of OperationResult is always nil
uri = "https://host.com/folder/pic.png"
result = client.delete_blob uri
result.success #=> Whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> nil
uri:string => nilDeletes the blob if it exist, else does nothing
Raises exception if the operation fails
uri = "https://host.com/folder/pic.png"
client.delete_blob! uri
The bucket instance is able to perform CRUD operations on blobs it owns.
Bang methods (methods that end with !) do not return OperationResult. Instead, they raise error if they fail.
In this section, we assume we obtain a Bucket instance from the Client instance using the get_bucket method.
bucket = client.get_bucket! "first-bucket"
bucket #=> bucket instance used to illustrate the examples below
key:string => OperationResult<array<byte>>Get blob as byte array
value of the OperationResult is the blob's byte array, if the operation succeeds
result = bucket.get_blob "image.png"
result.success #=> Whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> the byte array of the blob
key:string => array<byte>Get blob as byte array
Raises exception if the operation fails.
img = bucket.get_blob! "image.png"
IO.binwrite "image.png", img
key:string => booleanChecks if the blob with the given key exist.
exist = bucket.exist_blob "image.png"
exist #=> true if exist, false if it does not exist
payload:byte<array>,key:string => Operation<string>Create blob with payload. Fails if blob already exist.
value of OperationResult will return URI of the created blob if operations succeeded
img = IO.binread "image.png"
result = bucket.create_blob img, "image.png"
result.success #=> Whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> URI of the blob
payload:array<byte>,key:string => stringCreate blob with payload. Fails if blob already exist.
Raises exception if operation fails
img = IO.binread "image.png"
uri = bucket.create_blob! img, "image.png"
uri #=> URI of the created blob
payload:array<byte>, key:string => OperationResult<string>Updates the blob with new payload. Fails if blob does not exist
value of OperationResult will return URI of the created blob if operations succeeded
img = IO.binread "image.png"
result = bucket.update_blob img, "image.png"
result.success #=> whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> URI of the blob
payload:array<byte>, key:string => stringUpdates the blob with new payload. Fails if blob does not exist
Raises exception if the operation fails
img = IO.binread "image.png"
result = bucket.update_blob!(img, "image.png")
result #=> URI of updated blob
payload:array<byte>, key:string => OperationResult<string>Creates a new blob with payload if blob does not exist. Updates blob with new payload if blob exist
value of OperationResult will return URI of the created blob if operations succeeded
img = IO.binread "image.png"
result = bucket.put_blob(img, "image.png")
result.success #=> whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> URI of the blob
payload:array<byte>, key:string => stringCreates a new blob with payload if blob does not exist. Updates blob with new payload if blob exist
Raises exception if operation fails
img = IO.binread "image.png"
uri = bucket.put_blob! img, "image.png"
uri #=> uri of the blob
key:string => Operation<nil>Deletes a blob. Fails if the blob does not exist. To prevent this behaviour, use delete_blob_if_exist method
value of OperationResult will always return nil
result = bucket.delete_blob "image.png"
result.success #=> whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> nil
key:string => nilDeletes a blob. Fails if the blob does not exist. To prevent this behaviour, use delete_blob_if_exist method
Raises exception if the operation fails
bucket.delete_blob! "image.png"
key:string => Operation<nil>Deletes a blob if it exist.
value of OperationResult will always return nil
result = bucket.delete_blob_if_exist "image.png"
result.success #=> whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> nil
key:string => nilDeletes a blob if it exist.
Raises exception if the operation fails
bucket.delete_blob_if_exist! "image.png"
The bucket instance is able to perform CRUD operations on blobs it owns, via the full URI of the blob.
Bang methods (methods that end with !) do not return OperationResult. Instead, they raise error if they fail.
In this section, we assume we obtain a Bucket instance from the Client instance using the get_bucket! method.
bucket = client.get_bucket! "first-bucket"
bucket #=> bucket instance used to illustrate the examples below
uri:string => OperationResult<array<byte>>Get blob in target URI as byte array
value of the OperationResult is the blob's byte array, if the operation succeeds
result = bucket.get_blob_with_uri "https://domain.com/bucket/binary.ext"
result.success #=> Whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> the byte array of the blob
key:string => array<byte>Get blob in target URI as byte array
Raises exception if the operation fails.
img = bucket.get_blob_with_uri! "https://domain.com/bucket/binary.ext"
IO.binwrite "image.png", img
uri:string => booleanChecks if the blob with the given URI exist.
exist = bucket.exist_blob_with_uri "https://domain.com/bucket/binary.ext"
exist #=> true if exist, false if it does not exist
payload:array<byte>, uri:string => OperationResult<string>Updates the blob with new payload to the uri. Fails if blob does not exist
value of OperationResult will return URI of the created blob if operations succeeded
img = IO.binread "image.png"
result = bucket.update_blob_with_uri img, "https://domain.com/bucket/binary.ext"
result.success #=> whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> URI of the blob
payload:array<byte>, uri:string => stringUpdates the blob with new payload. Fails if blob does not exist
Raises exception if the operation fails
img = IO.binread "image.png"
result = bucket.update_blob_with_uri! img, "https://domain.com/bucket/binary.ext"
result #=> URI of updated blob
payload:array<byte>, uri:string => OperationResult<string>Creates a new blob with payload if blob does not exist. Updates blob with new payload if blob exist
value of OperationResult will return URI of the created blob if operations succeeded
img = IO.binread "image.png"
result = bucket.put_blob_with_uri img, "https://domain.com/bucket/binary.ext"
result.success #=> whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> URI of the blob
payload:array<byte>, uri:string => stringCreates a new blob with payload if blob does not exist. Updates blob with new payload if blob exist
Raises exception if operation fails
img = IO.binread "image.png"
uri = bucket.put_blob_with_uri! img, "https://domain.com/bucket/binary.ext"
uri #=> uri of the blob
uri:string => Operation<nil>Deletes a blob in the uri. Fails if the blob does not exist. To prevent this behaviour, use delete_blob_if_exist_with_uri method
value of OperationResult will always return nil
result = bucket.delete_blob_with_uri "https://domain.com/bucket/binary.ext"
result.success #=> whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> nil
uri:string => nilDeletes a blob. Fails if the blob does not exist. To prevent this behaviour, use delete_blob_if_exist_with_uri! method
Raises exception if the operation fails
bucket.delete_blob_with_uri! "https://domain.com/bucket/binary.ext"
uri:string => Operation<nil>Deletes a blob in the uri if it exist.
value of OperationResult will always return nil
result = bucket.delete_blob_if_exist_with_uri "https://domain.com/bucket/binary.ext"
result.success #=> whether the operation succeeded
result.code #=> Status Code of the operation
result.message #=> Error message if it failed
result.value #=> nil
uri:string => nilDeletes a blob in the uri if it exist.
Raises exception if the operation fails
bucket.delete_blob_if_exist_with_uri! "https://domain.com/bucket/binary.ext"
After checking out the repo, run bin/setup to install dependencies.
Then, run bundle exec rspec to run the unit tests, or if you are on RubyMime, the run configuration of
Unit Test is configured. You can run the test by selecting the Unit Test configuration.
To run the integration test with the cloud, please provide your cloud credentials in a .env file. The
credentials required are:
| Env Variable | Description |
|---|---|
| AWS_ID | The AWS access key ID |
| AWS_SECRET | The AWS secret |
| AWS_REGION | The 3region of the AWS bucket |
| DO_ID | Digital Ocean Spaces access key ID |
| DO_SECRET | Digital Ocean Spaces secret |
| DO_REGION | Digital Ocean Spaces region |
| AZURE_ACC_ID | The Azure Blob Storage account name |
| AZURE_KEY | The Azure Blob Storage secret |
| GOOGLE_ID | The project ID for the Blob Storage account |
| GOOGLE_KEY | The content of the secret JSON file in 1 line |
After setting up the .env file, you can now run the command bundle exec rspec ./integration.
Alternatively, the gitlab ci has prepared a set of blob storage account online to run integration test. You can activate the test by pushing a commit to your branch if setting up the environment it too complicated.
To install this gem onto your local machine, run bundle exec rake install.
Bug reports and pull requests are welcome on GitLab at https://gitlab.com/ruby-gem/bucket_client. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
The gem is available as open source under the terms of the MIT License.
Everyone interacting in the BucketClient project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.
FAQs
Unknown package
We found that bucket_client demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.