= Windows Azure Storage library — simple gem for accessing WAZ‘s Storage REST API A simple implementation of Windows Azure Storage API for Ruby, inspired by the S3 gems and self experience of dealing with queues. The major goal of the whole gem is to enable ruby developers [like me =)] to leverage Windows Azure Storage features and have another option for cloud storage.
The whole gem is implemented based on Microsoft's specs from the communication and underlying service description and protocol (REST). The API is for ruby developers built by a ruby developer. I'm trying to follow idioms, patterns and fluent type of doing APIs on Ruby.
This work isn't related at all with StorageClient Sample shipped with Microsoft SDK and written in .NET, the whole API is based on my own understanding, experience and values of elegance and ruby development.
Full documentation for the gem is available at http://waz-storage.heroku.com
=== How does this differ from waz-queues and waz-blobs work? Well, this is a sum up of the whole experience of writing those gems and getting them to work together to simplify end user experience. Although there're some breaking changes, it's pretty backward compatible with existing gems.
=== What's new on v1.1.4?
=== What's new on v1.1.2?
The main difference with the original library is the adding of the method "railsetag". This method add to any blob object an "x_ms_meta_railsetag" header, its value is calculated as the rails etag way (Digest::MD5.hexdigest(file))
By doing so we can easily know if a locale copy of a file is up-to-date on Azure CDN. This is very much used in the "waz-sync" gem which purpose is to sync file between local storage and Azure.
=== What's new on the v1.1 version? [thanks to smarx]
=== What's new on the v1.0.6 version? [thanks to hermes.logicalbricks]
=== What's new on the version version?
=== What's new on the v1.0.3 version?
=== What's new on the 1.0.2 version?
=== What's new on the 1.0.1 version?
=== What's new on the 1.0 version?
=== What's new on the 0.5.81 version?
=== What's new on the 0.5.8 version?
=== What's new on the 0.5.7 version?
=== What's new on the 0.5.6 version?
== Getting started sudo gem install waz-storage --source http://gemcutter.org
== Basic Configuration One of the major changes from the waz-queues and waz-blobs APIs was the ability to set a single storage account (99% of the cases) to be used on your code just once. Since Windows Azure Storage Services are all related to a single account that includes Tables, Queues and Blobs, I've changed the API in order to make ease that administration by requiring you to set the configuration just once:
require 'waz-queues' #=> if you want to use queues
require 'waz-blobs' #=> if you want to use blobs (or you can include both for using both)
WAZ::Storage::Base.establish_connection!(:account_name => account_name,
:access_key => access_key)
As you can see you, the way to get started it pretty simple, just include the establish_connection! call on your application bootstrapper, for example if you are doing a sinatra-rb application it can be
configure :production do
# here is were you hook up with WAZ Storage Stuff.
end
That's all you need to get started and running your Windows Azure Code on a sinatra application.
== Usage: Queues Windows Azure Queues are extremely powerful for asyc programming, they are very useful when you don’t need "always consistent" data operations allowing you to enqueue and process on background. It’s important to mention that being WAZ-Queues a buffered message system, you should design for idempotent operations given the fact that there’s no guarantee that a message can be repeated.
The implementation of the Windows Azure Queues REST API available online at http://msdn.microsoft.com/en-us/library/dd179363.aspx is fully covered here.
Here's a quick getting started of the usage of the API:
WAZ::Storage::Base.establish_connection!(:account_name => account_name,
:access_key => access_key)
# NEW! Now you can do ensure and it will return the queue, first trying to retrieve it
# and then if it doesn't exist, will created it
queue = WAZ::Queues::Queue.ensure('my-queue')
# excepts that the metadata for the queue changes this method behaves as PUT (create/replace)
# remarks: it performs a validation whether metadata changed or not (if changed HTTP 409 conflict)
queue = WAZ::Queues::Queue.create('my-queue')
10.times do |m|
# enqueue a receives string. Message content can be anything up to 8KB
# you can serialize and send anything that serializes to UTF-8 string (JSON, XML, etc)
queue.enqueue!("message##{m}")
end
while(queue.size > 0) do
# Since WAZ implements the peek lock pattern we are locking messages (not dequeuing)
# it has two parameters how many messages and for how long they are locked
messages = queue.lock(10)
puts "dequeued message: #{messages.size}"
# deletes the message from the queue so other clients do not pick it after
# visibility time out expires
messages.each {|m| m.destroy!}
end
It's pretty intuitive, but full documentation (RDoc) is available for the API for further reference.
== Usage: Blobs The blobs implementation inside this gem is fully compliant with the spec available at http://msdn.microsoft.com/en-us/library/dd135733.aspx. The Windows Azure Blobs REST API isn't fully covered here (see TODO's for more information). It's pretty usable and stable right now, I've been doing lot of testing around and it works seamlessly with the current Windows Azure implementation.
require 'waz-blobs'
WAZ::Storage::Base.establish_connection!(:account_name => account_name,
:access_key => access_key)
# creates a container
container = WAZ::Blobs::Container.create('my-container')
# stores a blob with custom properties (metadata)
blob = container.store('my_blob.txt',
'this is the content of my blob',
'plain/text',
{:x_ms_meta_Custom_Property => "custom_value" })
# return a specific blob from a container
blob = container['my_blob.txt']
# retrieves a blob value
blob.value
It's pretty intuitive, but full documentation (RDoc) is available for the API for further reference.
=== Using Blob with Shared Acess Signature This feature allow to access a blob only with the information of the blob's name, the container, and the Shared Access Signature. If you don't have permission to perform an action (for example: list the blobs, create a new blob or something like that) you will receive an exception. If you want more details about this: http://msdn.microsoft.com/en-us/library/ee395415.aspx
require 'waz-blobs'
WAZ::Storage::Base.establish_connection!(:account_name=> 'mysharedblob',
:use_sas_auth_only=> true,
:sharedaccesssignature =>"?se=XXXXXX&sr=c&si=XXXXX&sig=XXXXXXXXXXXXXXXXXXXXX")
container = WAZ::Blobs::Container.new(:name=>'container')
blob = container.store('myfile.txt',File.read('fileofthecontent.txt'),'plain/text')
The feature was only tested storing a blob.
== Usage: Tables The tables implementation inside this gem is fully compliant with the spec available at http://msdn.microsoft.com/en-us/library/dd179423.aspx. The Windows Azure Tables REST API is fully covered here.
require 'waz-storage'
require 'waz-tables'
WAZ::Storage::Base.establish_connection!(:account_name => account_name,
:access_key => access_key)
# creates a new table
table = WAZ::Tables::Table.create('my-table')
# list available tables
# (returns a maximum of 1,000 items at one time (more details here: http://msdn.microsoft.com/en-us/library/dd135718.aspx)
tables = WAZ::Tables::Table.list
# more tables
WAZ::Tables::Table.list(tables.continuation_token)
# get a specific table, returning nil when the specified table is not found
my_table = WAZ::Tables::Table.find('my-table')
# table properties
my_table.name
my_table.url
# delete a table
my_table.destroy!
# Entity operations using the Table service
# get the existing service instance
service = WAZ::Tables::Table.service_instance
# define a new entity
entity = { :address => 'Mountain View',
:age => 23,
:amount_due => 200.23,
:binary_data => File.open(__FILE__),
:customer_code => 'aaaaaaaa-bbbb-cccc-dddd-aaaabbbbcccc',
:customer_since => Time.now.utc,
:is_active => true,
:num_of_orders => 255,
:partition_key => 'customer',
:row_key => "myRowKey#{rand(2000000).to_s}",
:Timestamp => Time.now.utc }
# inserts a new entity
service.insert_entity('customer_table', entity)
# retrieves all entities from a table
# (returns a maximum of 1,000 items at one time (more details here: http://msdn.microsoft.com/en-us/library/dd135718.aspx)
# Remarks on development storage it retrieves all items instead the first 1,000
entities = service.query('customer_table')
# retrieves more entities providing the obtained continuation_token
service.query('customer_table', {:continuation_token => entities.continuation_token} )
# retrieves all records that match with the specified query but only the first fifteen rows
service.query('customer_table', {:expression => "(PartitionKey eq 'customer') and (Age eq 23)", :top => 15} )
# get an existing entity by its partion_key and row_key
entity = service.get_entity('customer_table', 'customer', 'rowKey1')
# updates an entity
entity[:age] = 90
service.update_entity('customer_table', new_entity)
# merges an entity (more details here: http://msdn.microsoft.com/en-us/library/dd179392.aspx)
entity[:age] = 20
service.merge_entity('customer_table', new_entity)
# deletes an entity
service.delete_entity('customer_table', 'customer', 'rowKey1')
It's pretty intuitive, but full documentation (RDoc) is available for the API for further reference.
== Usage: Contextual Connection Handling Sometimes while you are building a web application you may require handling different storage account but contextualized. The sample that comes to my mind is something like a Storage Explorer or Account Monitor for WAZ.
That is why? I've added a new way of handling a stack-based contextual connection handling. The usage is pretty simple:
WAZ::Storage::Base.establish_connection(options) do
container = WAZ::Blobs::Container.find('container-name')
blob = container['blob-name']
blob.destroy!
end
As it is described on the example above, there's a new way of establishing a connection and use it on a given block. The whole implementation is stack based, and will let you scope your context for some rare cases where you have another account.
Disclaimer: Moving objects across context isn't contemplated yet, and if you try to do changes among scopes you will get to some wired Windows Azure Errors regarding objects that may not exist.
=== Remarks Windows Azure Storage API works flawlessly from Heroku and custom ruby hosting deployments on EC2, as far as I tested it. You can leverage the storage services without the need of having to write the application on .NET or hosting your application on Windows Azure.
The documentation and implementation exposed here is for the pre-release version and is subject to change on the future.
=== TODO's As far as users start using it, I'll be building a backlog and probably handling a wish-list of features, but right now I've the following TODO's already enqueued for further releases of the waz-storage API.
-Generate a sample application to better show the usage.
The things listed above do not represent any sort of priority, or the order they are going to be tackled. It's just a list.
== Meta
Written by Johnny G. Halife (johnny.halife at me dot com)
contributed by: Ezequiel Morito (http://twitter.com/ezequielm), Juan Pablo Garcia (http://twitter.com/jpgd), Steve Marx (http://twitter.com/smarx), G. Montard (http://github.com/gmontard)
Released under the MIT License: http://www.opensource.org/licenses/mit-license.php
FAQs
Unknown package
We found that waz-storage demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.