Package storage provides an easy way to work with Google Cloud Storage.
Google Cloud Storage stores data in named objects, which are grouped into buckets.
More information about Google Cloud Storage is available at
See https://pkg.go.dev/cloud.google.com/go for authentication, timeouts,
connection pooling and similar aspects of this package.
To start working with this package, create a Client:
The client will use your default application credentials. Clients should be
reused instead of created as needed. The methods of Client are safe for
concurrent use by multiple goroutines.
You may configure the client by passing in options from the google.golang.org/api/option
package. You may also use options defined in this package, such as WithJSONReads.
If you only wish to access public data, you can create
an unauthenticated client with
To use an emulator with this library, you can set the STORAGE_EMULATOR_HOST
environment variable to the address at which your emulator is running. This will
send requests to that address instead of to Cloud Storage. You can then create
and use a client as usual:
Please note that there is no official emulator for Cloud Storage.
A Google Cloud Storage bucket is a collection of objects. To work with a
bucket, make a bucket handle:
A handle is a reference to a bucket. You can have a handle even if the
bucket doesn't exist yet. To create a bucket in Google Cloud Storage,
Note that although buckets are associated with projects, bucket names are
global across all projects.
Each bucket has associated metadata, represented in this package by
BucketAttrs. The third argument to BucketHandle.Create allows you to set
the initial BucketAttrs of a bucket. To retrieve a bucket's attributes, use
An object holds arbitrary data as a sequence of bytes, like a file. You
refer to objects using a handle, just as with buckets, but unlike buckets
you don't explicitly create an object. Instead, the first time you write
to an object it will be created. You can use the standard Go io.Reader
and io.Writer interfaces to read and write object data:
Objects also have attributes, which you can fetch with ObjectHandle.Attrs:
Listing objects in a bucket is done with the BucketHandle.Objects method:
Objects are listed lexicographically by name. To filter objects
lexicographically, [Query.StartOffset] and/or [Query.EndOffset] can be used:
If only a subset of object attributes is needed when listing, specifying this
subset using Query.SetAttrSelection may speed up the listing process:
Both objects and buckets have ACLs (Access Control Lists). An ACL is a list of
ACLRules, each of which specifies the role of a user, group or project. ACLs
are suitable for fine-grained control, but you may prefer using IAM to control
access at the project level (see Cloud Storage IAM docs.
To list the ACLs of a bucket or object, obtain an ACLHandle and call ACLHandle.List:
You can also set and delete ACLs.
Every object has a generation and a metageneration. The generation changes
whenever the content changes, and the metageneration changes whenever the
metadata changes. Conditions let you check these values before an operation;
the operation only executes if the conditions match. You can use conditions to
prevent race conditions in read-modify-write operations.
For example, say you've read an object's metadata into objAttrs. Now
you want to write to that object, but only if its contents haven't changed
since you read it. Here is how to express that:
You can obtain a URL that lets anyone read or write an object for a limited time.
Signing a URL requires credentials authorized to sign a URL. To use the same
authentication that was used when instantiating the Storage client, use
You can also sign a URL without creating a client. See the documentation of
SignedURL for details.
A type of signed request that allows uploads through HTML forms directly to Cloud Storage with
temporary permission. Conditions can be applied to restrict how the HTML form is used and exercised
by a user.
For more information, please see the XML POST Object docs as well
as the documentation of BucketHandle.GenerateSignedPostPolicyV4.
If the GoogleAccessID and PrivateKey option fields are not provided, they will
be automatically detected by BucketHandle.SignedURL and
BucketHandle.GenerateSignedPostPolicyV4 if any of the following are true:
Detecting GoogleAccessID may not be possible if you are authenticated using a
token source or using option.WithHTTPClient. In this case, you can provide a
service account email for GoogleAccessID and the client will attempt to sign
the URL or Post Policy using that service account.
To generate the signature, you must have:
Errors returned by this client are often of the type googleapi.Error.
These errors can be introspected for more information by using errors.As
with the richer googleapi.Error type. For example:
Methods in this package may retry calls that fail with transient errors.
Retrying continues indefinitely unless the controlling context is canceled, the
client is closed, or a non-transient error is received. To stop retries from
continuing, use context timeouts or cancellation.
The retry strategy in this library follows best practices for Cloud Storage. By
default, operations are retried only if they are idempotent, and exponential
backoff with jitter is employed. In addition, errors are only retried if they
are defined as transient by the service. See the Cloud Storage retry docs
for more information.
Users can configure non-default retry behavior for a single library call (using
BucketHandle.Retryer and ObjectHandle.Retryer) or for all calls made by a
client (using Client.SetRetry). For example:
You can add custom headers to any API call made by this package by using
callctx.SetHeaders on the context which is passed to the method. For example,
to add a custom audit logging header:
This package includes support for the Cloud Storage gRPC API, which is currently
in preview. This implementation uses gRPC rather than the current JSON & XML
APIs to make requests to Cloud Storage. If you would like to try the API,
please contact your GCP account rep for more information. The gRPC API is not
yet generally available, so it may be subject to breaking changes.
To create a client which will use gRPC, use the alternate constructor:
If the application is running within GCP, users may get better performance by
enabling DirectPath (enabling requests to skip some proxy steps). To enable,
set the environment variable `GOOGLE_CLOUD_ENABLE_DIRECT_PATH_XDS=true` and add
the following side-effect imports to your application: