awscli-cwlogs

============= awscli-cwlogs

This awscli plugin provides the pull, push and filter commands to access AWS CloudWatch Logs service.

The awscli-cwlogs package works on Python versions:

• 2.6.5 and greater
• 2.7.x and greater
• 3.3.x and greater

---

Installation

The easiest way to install awscli-cwlogs is to use pip_::

$ pip install awscli-cwlogs

or, if you are not installing in a virtualenv::

$ sudo pip install awscli-cwlogs

If you have the awscli-cwlogs installed and want to upgrade to the latest version you can run::

$ pip install --upgrade awscli-cwlogs

This will install the awscli-cwlogs package as well as all dependencies, including awscli.

.. attention:: If you have awscli installed, installing awscli-cwlogs might upgrade or downgrade your awscli depending on whether the awscli version you have is older or newer than what the awscli-cwlogs depends on. After installing awscli-cwlogs, you can run pip install --upgrade awscli to upgrade your awscli though potentially the latest awscli might not work well with the plugin. If you want to isolate awscli-cwlogs plugin from your existing awscli, you may consider virtaulenv. Be careful that awscli by default stores its configuration to ~/.aws/config (or in %UserProfile%.aws\config on Windows), to also isolate the configuration, you can define a separate config file export AWS_CONFIG_FILE=/path/to/config_file.

---

Getting Started

Before using awscli-cwlogs plugin, you need to configure awscli <http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html>__ first.

Once that's done, to enable awscli-cwlogs, you can run::

$ aws configure set plugins.cwlogs cwlogs

The above command adds below section to your aws config file::

[plugins]
cwlogs = cwlogs

To verify if awscli-cwlogs plugin is installed and configured properly, you can run::

$ aws logs help

You will see the pull, push and fitler commands from available commands, otherwise it means the cwlogs plugin is not registered properly.

If you see ImportError: No module named cwlogs error, it means the cwlogs plugin is registered in config file, but the plugin is not installed.

^^^^^^^^ Examples ^^^^^^^^ ............. Push command ............. You can use aws logs push help to check supported options. The push command is used by CloudWatch Logs agent, check the CloudWatch Logs Agent Reference <https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/AgentReference.html>__ to see all supported options or if you want to keep the push command running.

1. Uploading a single log event to CloudWatch Logs service. The log group and log stream get created automatically if they don't exist.

::

echo "Hello World" | aws logs push --log-group-name MyLogGroup --log-stream-name MyLogStream

2) The following push command pushes log events from a syslog file to log stream which is specified by /var/log/syslog and myhost1 and exits after pushing all log events. This command doesn't push the incremental log events. To achieve that, use tail -f file | aws logs push ....

::

cat /var/log/kernel.log | aws logs push --log-group-name /var/log/syslog --log-stream-name myhost1 --datetime-format '%b %d %H:%M:%S' --time-zone LOCAL --encoding ascii

3) The following push command pushes log events from multiple files based on configuration file. The initial_position determines where to start if the state of file is not available.

::

aws logs push --config-file push.cfg

::

[general]
state_file = push-state
[logstream-messages]
datetime_format = %b %d %H:%M:%S
time_zone = LOCAL
file = /var/log/messages
file_fingerprint_lines = 1
log_group_name = /var/log/messages
log_stream_name = {hostname}
initial_position = start_of_file
encoding = utf_8
buffer_duration = 5000
[logstream-system.log]
datetime_format = %b %d %H:%M:%S
time_zone = UTC
file = /var/log/system.log
file_fingerprint_lines = 1-3
log_group_name = /var/log/system.log
log_stream_name = {hostname}
initial_position = end_of_file
encoding = ascii
buffer_duration = 10000

............. Pull command ............. You can use aws logs pull help to check supported options.

1. The following pull command pulls log events starting at 2014-01-23T00:00:00Z from one log stream which is specified by website1/access_log and webhost-001 and exits after pulling all log events.

::

aws logs pull --log-group-name website1/access_log --log-stream-name webhost-001 --start-time 2014-01-23T00:00:00Z

2) When invoked with the --end-time option, the following pull command pulls all log events between 2014-01-23T00:00:00Z (inclusive) and 2014-01-23T01:00:00Z (not inclusive).

::

aws logs pull --log-group-name website1/access_log --log-stream-name webhost-001 --start-time 2014-01-23T00:00:00Z --end-time 2014-01-23T01:00:00Z

3) When invoked with the --follow option, the following pull command does not exit after pulling all log events, but polls continuously for new log events.

::

aws logs pull --log-group-name website1/access_log --log-stream-name webhost-001 --start-time 2014-01-23T00:00:00Z --follow

4) When invoked with the --output-format option, the following pull command only outputs the message field. By default, the output format is "{timestamp} {message}". Ingestion time can be included with "{timestamp} {ingestionTime} {message}".

::

aws logs pull --log-group-name website1/access_log --log-stream-name webhost-001 --start-time 2014-01-23T00:00:00Z --output-format "{message}"

............... Filter command ............... See this AWS developer guide <http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/SearchDataFilterPattern.html>__.

.. _pip: ht`tp://www.pip-installer.org/en/latest/