jldaws

Overview

This package contains a collection of Amazon Web Service related scripts e.g.

• jldsqsjob : wait for stdin trigger, dequeue msg(s) from SQS Q#1, send to stdout, requeue to SQS Q#2, delete msg(s) from Q#1
• jlds3upload : automated file upload to S3
• jlds3download : automated file download from S3
• jldrxsqs : receive from an SQS queue to JSON/string stdout (with optional stdin trigger)
• jldtxsqs : transmit on SQS JSON/string from stdin
• jldleader: distributed leader election protocol manager
• jlds3up : simple file upload to s3 with functionality to delete 'old' files
• jlds3list : listing files in an S3 bucket
• jldsdb: list, create and delete operations on SDB
• jldsdbget: retrieve entries in SDB
• jldsdbinsert: insert records in SDB
• jldsdbbatchinsert: batch insert records in SDB from file

The philosophy behind these scripts is:

• simplicity : each script only does 1 thing
• reporting : status & error reporting through stderr and (optionally) JSON stdout
• data driven : object in from src ==> object out to dst
• data flow : piping through stdin / stdout

Scripts can easily be "composed" through standard Linux piping :

jldrxsqs @config1.txt | some_other_command | jldtxsqs @config2.txt

The data format used is JSON.

Configuration

Can be performed through options on the command line or using a file (use a leading @).

History

Changes history can be found at the project page: https://github.com/jldupont/jldaws