Azure Queue storage is a service for storing large numbers of messages that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS. A single queue message can be up to 64 KiB in size, and a queue can contain millions of messages, up to the total capacity limit of a storage account.
Common uses of Queue storage include:
Source code | Package (PyPI) | Package (Conda) | API reference documentation | Product documentation | Samples
Install the Azure Storage Queues client library for Python with pip:
pip install azure-storage-queue
If you wish to create a new storage account, you can use the Azure Portal, Azure PowerShell, or Azure CLI:
# Create a new resource group to hold the storage account -
# if using an existing resource group, skip this step
az group create --name my-resource-group --location westus2
# Create the storage account
az storage account create -n my-storage-account-name -g my-resource-group
The Azure Storage Queues client library for Python allows you to interact with three types of resources: the storage account itself, queues, and messages. Interaction with these resources starts with an instance of a client. To create a client object, you will need the storage account's queue service endpoint URL and a credential that allows you to access the storage account:
from azure.storage.queue import QueueServiceClient
service = QueueServiceClient(account_url="https://<my-storage-account-name>.queue.core.windows.net/", credential=credential)
You can find the storage account's queue service URL using the Azure Portal, Azure PowerShell, or Azure CLI:
# Get the queue service URL for the storage account
az storage account show -n my-storage-account-name -g my-resource-group --query "primaryEndpoints.queue"
The credential parameter may be provided in a number of different forms, depending on the type of
authorization you wish to use:
To use a shared access signature (SAS) token,
provide the token as a string. If your account URL includes the SAS token, omit the credential parameter.
You can generate a SAS token from the Azure Portal under "Shared access signature" or use one of the generate_sas()
functions to create a sas token for the storage account or queue:
from datetime import datetime, timedelta
from azure.storage.queue import QueueServiceClient, generate_account_sas, ResourceTypes, AccountSasPermissions
sas_token = generate_account_sas(
account_name="<storage-account-name>",
account_key="<account-access-key>",
resource_types=ResourceTypes(service=True),
permission=AccountSasPermissions(read=True),
start=datetime.utcnow(),
expiry=datetime.utcnow() + timedelta(hours=1)
)
queue_service_client = QueueServiceClient(account_url="https://<my_account_name>.queue.core.windows.net", credential=sas_token)
To use a storage account shared key (aka account key or access key), provide the key as a string. This can be found in the Azure Portal under the "Access Keys" section or by running the following Azure CLI command:
az storage account keys list -g MyResourceGroup -n MyStorageAccount
Use the key as the credential parameter to authenticate the client:
from azure.storage.queue import QueueServiceClient
service = QueueServiceClient(account_url="https://<my_account_name>.queue.core.windows.net", credential="<account_access_key>")
To use an Azure Active Directory (AAD) token credential, provide an instance of the desired credential type obtained from the azure-identity library. For example, DefaultAzureCredential can be used to authenticate the client.
This requires some initial setup:
Use the returned token credential to authenticate the client:
from azure.identity import DefaultAzureCredential
from azure.storage.queue import QueueServiceClient
token_credential = DefaultAzureCredential()
queue_service_client = QueueServiceClient(
account_url="https://<my_account_name>.queue.core.windows.net",
credential=token_credential
)
Depending on your use case and authorization method, you may prefer to initialize a client instance with a storage
connection string instead of providing the account URL and credential separately. To do this, pass the storage
connection string to the client's from_connection_string class method:
from azure.storage.queue import QueueServiceClient
connection_string = "DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=xxxx;EndpointSuffix=core.windows.net"
service = QueueServiceClient.from_connection_string(conn_str=connection_string)
The connection string to your storage account can be found in the Azure Portal under the "Access Keys" section or by running the following CLI command:
az storage account show-connection-string -g MyResourceGroup -n MyStorageAccount
The following components make up the Azure Queue Service:
The Azure Storage Queues client library for Python allows you to interact with each of these components through the use of a dedicated client object.
This library includes a complete async API supported on Python 3.5+. To use it, you must first install an async transport, such as aiohttp. See azure-core documentation for more information.
Async clients and credentials should be closed when they're no longer needed. These
objects are async context managers and define async close methods.
Two different clients are provided to interact with the various components of the Queue Service:
get_queue_client method.The following sections provide several code snippets covering some of the most common Storage Queue tasks, including:
Create a queue in your storage account
from azure.storage.queue import QueueClient
queue = QueueClient.from_connection_string(conn_str="<connection_string>", queue_name="myqueue")
queue.create_queue()
Use the async client to create a queue
from azure.storage.queue.aio import QueueClient
queue = QueueClient.from_connection_string(conn_str="<connection_string>", queue_name="myqueue")
await queue.create_queue()
Send messages to your queue
from azure.storage.queue import QueueClient
queue = QueueClient.from_connection_string(conn_str="<connection_string>", queue_name="myqueue")
queue.send_message("I'm using queues!")
queue.send_message("This is my second message")
Send messages asynchronously
import asyncio
from azure.storage.queue.aio import QueueClient
queue = QueueClient.from_connection_string(conn_str="<connection_string>", queue_name="myqueue")
await asyncio.gather(
queue.send_message("I'm using queues!"),
queue.send_message("This is my second message")
)
Receive and process messages from your queue
from azure.storage.queue import QueueClient
queue = QueueClient.from_connection_string(conn_str="<connection_string>", queue_name="myqueue")
response = queue.receive_messages()
for message in response:
print(message.content)
queue.delete_message(message)
# Printed messages from the front of the queue:
# >> I'm using queues!
# >> This is my second message
Receive and process messages in batches
from azure.storage.queue import QueueClient
queue = QueueClient.from_connection_string(conn_str="<connection_string>", queue_name="myqueue")
response = queue.receive_messages(messages_per_page=10)
for message_batch in response.by_page():
for message in message_batch:
print(message.content)
queue.delete_message(message)
Receive and process messages asynchronously
from azure.storage.queue.aio import QueueClient
queue = QueueClient.from_connection_string(conn_str="<connection_string>", queue_name="myqueue")
response = queue.receive_messages()
async for message in response:
print(message.content)
await queue.delete_message(message)
Optional keyword arguments that can be passed in at the client and per-operation level.
Use the following keyword arguments when instantiating a client to configure the retry policy:
retry_total=0 if you do not want to retry on requests. Defaults to 10.False.Other optional configuration keyword arguments that can be specified on the client or per-operation.
Client keyword arguments:
Per-operation keyword arguments:
headers={'CustomValue': value}Storage Queue clients raise exceptions defined in Azure Core.
This list can be used for reference to catch thrown exceptions. To get the specific error code of the exception, use the error_code attribute, i.e, exception.error_code.
This library uses the standard logging library for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level.
Detailed DEBUG level logging, including request/response bodies and unredacted
headers, can be enabled on a client with the logging_enable argument:
import sys
import logging
from azure.storage.queue import QueueServiceClient
# Create a logger for the 'azure.storage.queue' SDK
logger = logging.getLogger('azure.storage.queue')
logger.setLevel(logging.DEBUG)
# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)
# This client will log detailed information about its HTTP sessions, at DEBUG level
service_client = QueueServiceClient.from_connection_string("your_connection_string", logging_enable=True)
Similarly, logging_enable can enable detailed logging for a single operation,
even when it isn't enabled for the client:
service_client.get_service_stats(logging_enable=True)
Get started with our Queue samples.
Several Storage Queues Python SDK samples are available to you in the SDK's GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Storage Queues:
queue_samples_hello_world.py (async version) - Examples found in this article:
queue_samples_authentication.py (async version) - Examples for authenticating and creating the client:
queue_samples_service.py (async version) - Examples for interacting with the queue service:
queue_samples_message.py (async version) - Examples for working with queues and messages:
For more extensive documentation on Azure Queue storage, see the Azure Queue storage documentation on docs.microsoft.com.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
azure-core dependency to 1.30.0.This version and all future versions will require Python 3.8+. Python 3.7 is no longer supported.
services parameter has been added to the generate_account_sas API, which enables the ability to generate SAS
tokens to be used with multiple services. By default, the SAS token service scope will default to the current service.typing-extensions to >=4.6.0 to avoid potential TypeError with typing.TypeVar on
Python 3.12.AttributeError instead of ClientAuthenticationError when
using async OAuth credentials.audience as an optional keyword that can be specified on APIs that have a credential parameter. This
keyword only has an effect when the credential provided is of type TokenCredential.credential during client construction, the
__str__ of the object would be present in the exception message and therefore potentially logged.user_agent was being ignored on send_message, receive_message, receive_messages,
update_message, and peek_messages if client was configured for encryption.KeyError: 'sdk_moniker' in create_configuration.
NOTE: This is not an exported method and therefore should not be imported/called directly.azure-core minimum dependency to 1.28.0 and typing-extensions to 4.3.0.msrest dependency.typing-extensions>=4.0.1 as a dependency.isodate>=0.6.1 as a dependency.aio for installing optional async dependencies. Use pip install azure-storage-queue[aio] to install.read_timeout to 60 seconds for all clients.This version and all future versions will require Python 3.7+. Python 3.6 is no longer supported.
AzureNamedKeyCredential as a valid credential type.encryption_version
keyword (i.e. encryption_version='2.0').max_messages in receive_messages() to specify the maximum number of messages to receive from the queue.receive_messages() to explain iterator behavior and life-cycle.queue_samples_message.py (and async-equivalent) showcasing the use of max_messages in receive_messages().This version and all future versions will require Python 3.6+. Python 2.7 is no longer supported.
azure-core dependency to avoid inconsistent dependencies from being installed.Fixes
AccountName, AccountKey etc. in conn_str case insensitiveThreadPoolExecutor (#8955)New features
AzureSasCredential to allow SAS rotation in long living clients.New feature
receive_message on QueueClient to support receiving one message from queue (#14844, #14762)Notes
azure-core from azure-core<2.0.0,>=1.6.0 to azure-core<2.0.0,>=1.9.0 to get continuation_token attr on AzureError.Fixes
Notes
Fixes
Notes
StorageUserAgentPolicy is now replaced with the UserAgentPolicy from azure-core. With this, the custom user agents are now added as a prefix instead of being appended.New features
close() method to close the sockets opened by the client when using without a context manager.Breaking changes
QueueClient now accepts only account_url with mandatory a string param queue_name.
To use a queue_url, the method from_queue_url must be used.set_queue_access_policy has required parameter signed_identifiers.NoRetry policy has been removed. Use keyword argument retry_total=0 for no retries.NoEncodePolicy and NoDecodePolicy have been removed. Use message_encode_policy=None and message_decode_policy=None.QueueServiceClient and QueueClient
should be imported from azure.storage.queue.aioloopmax_concurrencyvalidate_contenttimeout etc.QueueMessage has had its parameters renamed from insertion_time, time_next_visible, expiration_time
to inserted_on, next_visible_on, expires_on, respectively.Logging has been renamed to QueueAnalyticsLogging.enqueue_message is now called send_message.azure.storage.queue and azure.storage.queue.aio only.generate_shared_access_signature methods on both QueueServiceClient and QueueClient have been replaced by module level functions generate_account_sas and generate_queue_sas.get_service_stats now returns a dictget_service_properties now returns a dict with keys consistent to set_service_propertiesNew features
ResourceTypes, and Services now have method from_string which takes parameters as a string.Fixes and improvements
Breaking changes
AccountPermissions, QueuePermissions have been renamed to
AccountSasPermissions, QueueSasPermissions respectively.__add__ and __or__ methods are removed.max_connections is now renamed to max_concurrency.New features
AccountSasPermissions, QueueSasPermissions now have method from_string which takes parameters as a string.Dependency updates
Adopted azure-core 1.0.0b3
pip install azure-core==1.0.0b2 azure-storage-queue==12.0.0b2
Breaking changes
marker parameter has been removed.by_page function that will return a secondary iterator of batches of results. This function supports a continuation_token parameter to replace the previous marker parameter.receive_messages operation:
by_page operation to receive messages in batches.New features
azure.storage.queue.aio.Dependency updates
Adopted azure-core 1.0.0b2
pip install azure-core==1.0.0b1 azure-storage-queue==12.0.0b1
Fixes and improvements
Version 12.0.0b1 is the first preview of our efforts to create a user-friendly and Pythonic client library for Azure Storage Queues. For more information about this, and preview releases of other Azure SDK libraries, please visit https://aka.ms/azure-sdk-preview1-python.
Breaking changes: New API design
Operations are now scoped to a particular client:
QueueServiceClient: This client handles account-level operations. This includes managing service properties and listing the queues within an account.QueueClient: The client handles operations within a particular queue. This includes creating or deleting that queue, as well as enqueueing and dequeueing messages.These clients can be accessed by navigating down the client hierarchy, or instantiated directly using URLs to the resource (account or queue). For full details on the new API, please see the reference documentation.
New message iterator, for receiving messages from a queue in a continuous stream.
New underlying REST pipeline implementation, based on the new azure-core library.
Client and pipeline configuration is now available via keyword arguments at both the client level, and per-operation. See reference documentation for a full list of optional configuration arguments.
Authentication using azure-identity credentials
New error hierarchy:
azure.core.exceptions.HttpResponseErrorResourceNotFoundError: The resource (e.g. queue, message) could not be found. Commonly a 404 status code.ResourceExistsError: A resource conflict - commonly caused when attempting to create a resource that already exists.ResourceModifiedError: The resource has been modified (e.g. overwritten) and therefore the current operation is in conflict. Alternatively this may be raised if a condition on the operation is not met.ClientAuthenticationError: Authentication failed.No longer have specific operations for get_metadata - use get_properties instead.
No longer have specific operations for exists - use get_properties instead.
Operations get_queue_acl and set_queue_acl have been renamed to get_queue_access_policy and set_queue_access_policy.
Operation put_message has been renamed to enqueue_message.
Operation get_messages has been renamed to receive_messages.
FAQs
Microsoft Azure Azure Queue Storage Client Library for Python
We found that azure-storage-queue demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.