volos-s3

volos-s3 connector

The volos-s3 connector lets you perform CRUD operations on an Amazon Web Services Simple Storage Service (S3) account.

What is the volos-s3 connector?

The volos-s3 connector maps S3 operations to RESTful API resources and query parameters. For example, a properly configured S3 connector returns a list of buckets, like this:

curl http://localhost:9058/buckets

and you get back a JSON response like this:

    "Buckets": [
        {
            "Name": "com.mycompany.bucket",
            "CreationDate": "2014-07-10T17:23:57.000Z"
        }
    ],
    "Owner": {
        "DisplayName": "jdoe",
        "ID": "<a long string>"
    }
}

Getting started

To use this connector you need two things:

• a correctly configured S3 connection, and
• an S3-to-REST mapping file.

Let's start by configuring a connection and testing it with the default mapping file. After that, we'll dive into the details of customizing the mapping file.

What do I need to do first?

This connector is a Node.js module. So, you must download and install Node.js on your system to use the connector. Follow the instructions at http://nodejs.org/download/.

Most of our examples use cURL, a utility for making HTTP requests to servers. We recommend you install cURL or have a REST tool like Postman handy.

How do I install the connector?

This connector is available through npm. Note that npm was installed when you installed Node.js.

1. Create a folder for the connector.
2. Go to that folder.
3. Enter: npm install volos-s3

How do I configure my AWS S3 connection?

The connector needs to know a little bit about your AWS S3 account before it can actually connect. You'll need this information to complete the config:

• acessKeyId - The access key ID for your Amazon Web Services S3 account.
• secretAccessKey - The secret access key for your Amazon Web Services S3 account.

Tip: Log in to your S3 account to find your security credentials, including AWS access keys.

Let's walk through the configuration steps:

Note: We're going to run a utility called vaultcli.js, which will encrypt your sensitive database login credentials. This utility is an npm module called avault, which was installed along with this connector. If you're curious, you can find it in the node_modules folder.

1. In a terminal, go to the folder where the volos-s3 module is installed.

2. Enter this command to encrypt your S3 credentials. Fill in your AWS access keys. Also, specify a name for the vault, which is the name the connector uses to load your encrypted credentials.

   ./node_modules/avault/vaultcli.js --verbose --value='{"accessKeyId":"my-aws-access-key-id", "secretAccessKey": "my-aws-secret-access-key"}' my-vault-name

3. Your connector folder now has two new files: store.js and keys.js. Check to make sure they're present.

4. Open the file server-http.js and change the first parameter of the vault.get() method to the name of the vault you created previously:

   vault.get('my-vault-name', function(profileString)

5. Start the HTTP server:

   node server-http

6. Test the server. In another terminal, or in a REST tool like Postman, enter this command:

   curl http://localhost:9058

7. This call returns usage information for the API:

{
    "usage": {
        "Commands": [
            "GET /buckets/:bucketid/object  Query Parameters: delimiter, encodingType, key",
            "PUT /buckets/:bucketid/object",
            "DELETE /buckets/:bucketid/object",
            "GET /buckets/:bucketid  Query Parameters: delimiter, encodingType, marker, prefix",
            "GET /buckets"
        ],
        "Common Optional Parameters": [
            "limit=<maxResults>",
            "expand=true"
        ]
    }
}

Note that there are some optional query parameters that you can use to filter the results: limit and expand.

• limit - Specifies the maximum number of records to fetch. For example: ?limit=10.
• expand - Retrieves expanded records. You can customize both the basic and expanded query strings in the mapping file. For example: ?expand=true. By default, this parameter is false.

Checkpoint

You've configured the S3 connector and verified that you have a valid connection. Next, we'll explain how to use the API.

Do I need to configure the connector?

The answer is "no". The file configurations.js contains the infomation that maps S3 bucket queries to well-defined RESTful API resources. You don't need to change this file, but, if you're curious, you can read it to see exactly how the REST API is constructed.

How do I use the API?

To use this connector's REST API, simply refer to the usage information shown previously. For example, to get a list of all of your buckets:

curl http://localhost:9056/buckets

To get a list of all the objects in a bucket, try this command:

curl http://localhost:9056/buckets/mybucketid

You'll get a response that looks something like this:

{
    "IsTruncated": false,
    "Marker": null,
    "Contents": [
        {
            "Key": "glyphicons_064_lightbulb.png",
            "LastModified": "2014-07-10T23:38:32.000Z",
            "ETag": "\"19e7761f59806fe3b74ab72edcfbf4ba\"",
            "Size": 1553,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "<myname>",
                "ID": "<a long string>"
            }
        },
        {
            "Key": "glyphicons_181_download_alt.png",
            "LastModified": "2014-07-10T23:38:35.000Z",
            "ETag": "\"2e801ecde8b8706e8c8ecda63ba3aca3\"",
            "Size": 1289,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "<myname>",
                "ID": "<a long string>"
            }
        },
    ...

To add an object to a bucket:

``curl -X PUT -H Content-Type: application/json http://localhost:9056/buckets//object?Key= -d {./myfile}