fluent-plugin-cloudwatch
Advanced tools
AWS CloudWatch input plugin.
This plugin is simple. Get metrics from cloudwatch to fluentd.
<source>
@type cloudwatch
tag cloudwatch
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET_KEY
cw_endpoint ENDPOINT
namespace [namespace]
statistics [statistics] (default: Average)
metric_name [metric name]
dimensions_name [dimensions_name]
dimensions_value [dimensions value]
period [period] (default: 300)
interval [interval] (default: 300)
delayed_start [bool] (default: false)
emit_zero [bool] (default: false)
</source>
<source>
@type cloudwatch
tag cloudwatch
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET_KEY
cw_endpoint monitoring.ap-northeast-1.amazonaws.com
namespace AWS/RDS
metric_name CPUUtilization,FreeStorageSpace,DiskQueueDepth,FreeableMemory,SwapUsage,ReadIOPS,ReadLatency,ReadThroughput,WriteIOPS,WriteLatency,WriteThroughput
dimensions_name DBInstanceIdentifier
dimensions_value rds01
</source>
<match cloudwatch>
@type copy
<store>
@type file
path /var/log/td-agent/test
</store>
</match>
2013-02-24T13:40:00+09:00 cloudwatch {"CPUUtilization":2.0}
2013-02-24T13:40:00+09:00 cloudwatch {"FreeStorageSpace":104080723968.0}
2013-02-24T13:39:00+09:00 cloudwatch {"DiskQueueDepth":0.002000233360558732}
2013-02-24T13:40:00+09:00 cloudwatch {"FreeableMemory":6047948800.0}
2013-02-24T13:40:00+09:00 cloudwatch {"SwapUsage":0.0}
2013-02-24T13:40:00+09:00 cloudwatch {"ReadIOPS":0.4832769510223807}
2013-02-24T13:40:00+09:00 cloudwatch {"ReadLatency":0.0}
2013-02-24T13:39:00+09:00 cloudwatch {"ReadThroughput":0.0}
2013-02-24T13:40:00+09:00 cloudwatch {"WriteIOPS":5.116069791857616}
2013-02-24T13:40:00+09:00 cloudwatch {"WriteLatency":0.004106280193236715}
2013-02-24T13:39:00+09:00 cloudwatch {"WriteThroughput":54074.40992132284}
<source>
@type cloudwatch
tag cloudwatch
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET_KEY
cw_endpoint monitoring.ap-northeast-1.amazonaws.com
namespace AWS/ELB
metric_name HealthyHostCount,HTTPCode_Backend_2XX,HTTPCode_Backend_3XX,HTTPCode_Backend_4XX,HTTPCode_Backend_5XX,HTTPCode_ELB_4XX,Latency,RequestCount,UnHealthyHostCount
dimensions_name LoadBalancerName
dimensions_value YOUR_ELB_NAME
</source>
2013-03-21T14:08:00+09:00 cloudwatch {"HealthyHostCount":2.0}
2013-03-21T14:08:00+09:00 cloudwatch {"HTTPCode_Backend_2XX":1.0}
2013-03-21T14:08:00+09:00 cloudwatch {"Latency":0.004025}
2013-03-21T14:08:00+09:00 cloudwatch {"RequestCount":1.0}
2013-03-21T14:09:00+09:00 cloudwatch {"UnHealthyHostCount":0.0}
<source>
@type cloudwatch
tag cloudwatch
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET_KEY
cw_endpoint monitoring.ap-northeast-1.amazonaws.com
namespace AWS/EC2
metric_name CPUUtilization,FreeStorageSpace,DiskQueueDepth,FreeableMemory,SwapUsage,ReadIOPS,ReadLatency,ReadThroughput,WriteIOPS,WriteLatency,WriteThroughput
dimensions_name InstanceId
dimensions_value YOUR_INSTANCE_ID
</source>
2013-02-25T00:44:00+09:00 cloudwatch {"CPUUtilization":1.58}
2013-02-25T00:44:00+09:00 cloudwatch {"DiskReadBytes":0.0}
2013-02-25T00:44:00+09:00 cloudwatch {"DiskReadBytes":0.0}
2013-02-25T00:44:00+09:00 cloudwatch {"DiskWriteBytes":0.0}
2013-02-25T00:44:00+09:00 cloudwatch {"DiskWriteOps":0.0}
2013-02-25T00:44:00+09:00 cloudwatch {"NetworkIn":95183.0}
2013-02-25T00:44:00+09:00 cloudwatch {"NetworkOut":95645.0}
@type cloudwatch
tag cloudwatch
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET_KEY
cw_endpoint monitoring.ap-northeast-1.amazonaws.com
namespace AWS/DynamoDB
metric_name ConsumedReadCapacityUnits,ConsumedWriteCapacityUnits
dimensions_name TableName
dimensions_value some_dd_tablename
statistics Sum
interval 300
period 300
2013-04-11 15:13:00 +0900 cloudwatch {"ConsumedReadCapacityUnits":8271.5}
2013-04-11 15:13:00 +0900 cloudwatch {"ConsumedWriteCapacityUnits":2765.5}
Note: Billing requires the us-east-1 endpoint
@type cloudwatch
tag cloudwatch
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET_KEY
cw_endpoint monitoring.us-east-1.amazonaws.com
namespace AWS/Billing
metric_name EstimatedCharges
dimensions_name Currency
dimensions_value USD
statistics Average
interval 7200
period 21600
2013-06-10 02:03:00 +0900 cloudwatch {"EstimatedCharges_in_USD":"543.175"}
2013-06-10 04:03:00 +0900 cloudwatch {"EstimatedCharges_in_USD":"550.39"}
@type cloudwatch
tag cloudwatch
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET_KEY
cw_endpoint monitoring.us-east-1.amazonaws.com
namespace AWS/StorageGateway
metric_name CacheHitPercent,CachePercentUsed
dimensions_name GatewayId,GatewayName
dimensions_value sgw-XXXXXXXX,mygateway
statistics Average
2014-01-20 20:12:00 +0900 cloudwatch: {"CacheHitPercent":0.0}
2014-01-20 20:12:00 +0900 cloudwatch: {"CachePercentUsed":95.15519175634687}
metric_name format is allowed as below.
MetricNameMetricName:StatsticsFor example, this configuration fetches "Sum of RequestCount" and "Average of Latancy".
metric_name RequestCount,Latency:Average
statistics Sum
When config delayed_start is set true, plugin startup will be delayed in random seconds(0 ~ interval).
unit: seconds.
flunet-plugin-cloudwatch gets metrics between now and period × 10 sec ago, and pick a latest value from that.
But the latest metric is insufficient for statistics Sum.
If offset is specified, fluent-plugin-cloudwatch gets metrics between offset sec ago and older.
If emit_zero is true and cloudwatch datapoint is empty, fluent-plugin-cloudwatch emits 0 instead of warn log "datapoint is empty".
record_attr is an optional value injected into each records.
<source>
@type cloudwatch
tag cloudwatch.rds
aws_key_id YOUR_AWS_KEY_ID
aws_sec_key YOUR_AWS_SECRET_KEY
cw_endpoint monitoring.ap-northeast-1.amazonaws.com
namespace AWS/RDS
metric_name CPUUtilization,FreeStorageSpace
dimensions_name DBInstanceIdentifier
dimensions_value rds01
record_attr DBInstanceIdentifier:rds01
# JSON format
# record_attr {"DBInstanceIdentifier":"rds01"}
</source>
Example output.
2017-11-04T13:40:00+09:00 cloudwatch.rds {"CPUUtilization":2.0,"DBInstanceIdentifier":"rds01"}
2013-11-04T13:40:00+09:00 cloudwatch.rds {"FreeStorageSpace":104080723968.0,"DBInstanceIdentifier":"rds01"}
git checkout -b my-new-feature)git commit -am 'Added some feature')git push origin my-new-feature)FAQs
Unknown package
We found that fluent-plugin-cloudwatch demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.