Security News
New Python Packaging Proposal Aims to Solve Phantom Dependency Problem with SBOMs
PEP 770 proposes adding SBOM support to Python packages to improve transparency and catch hidden non-Python dependencies that security tools often miss.
aws-embedded-metrics
Advanced tools
Generate CloudWatch Metrics embedded within structured log events. The embedded metrics will be extracted so you can visualize and alarm on them for real-time incident detection. This allows you to monitor aggregated values while preserving the detailed event context that generated them.
Generate custom metrics across compute environments
Linking metrics to high cardinality context
Using the Embedded Metric Format, you will be able to visualize and alarm on custom metrics, but also retain the original, detailed and high-cardinality context which is queryable using CloudWatch Logs Insights. For example, the library automatically injects environment metadata such as Lambda Function version, EC2 instance and image ids into the structured log event data.
npm install aws-embedded-metrics
Important Versions 4.1.1+, 3.0.2+, 2.0.7+ are required for usage in Lambda with JSON log format. Using previous versions in such Lambda environments will lead to metric loss.
To get a metric logger, you can either decorate your function with a metricScope, or manually create and flush the logger.
Using the metricScope decorator without function parameters:
const { metricScope, Unit, StorageResolution } = require("aws-embedded-metrics");
const myFunc = metricScope(metrics =>
async () => {
metrics.putDimensions({ Service: "Aggregator" });
metrics.putMetric("ProcessingLatency", 100, Unit.Milliseconds, StorageResolution.Standard);
metrics.putMetric("Memory.HeapUsed", 1600424.0, Unit.Bytes, StorageResolution.High);
metrics.setProperty("RequestId", "422b1569-16f6-4a03-b8f0-fe3fd9b100f8");
// ...
});
await myFunc();
Using the metricScope decorator with function parameters:
const { metricScope, Unit, StorageResolution } = require("aws-embedded-metrics");
const myFunc = metricScope(metrics =>
async (param1: string, param2: number) => {
metrics.putDimensions({ Service: "Aggregator" });
metrics.putMetric("ProcessingLatency", 100, Unit.Milliseconds, StorageResolution.Standard);
metrics.putMetric("Memory.HeapUsed", 1600424.0, Unit.Bytes, StorageResolution.High);
metrics.setProperty("RequestId", "422b1569-16f6-4a03-b8f0-fe3fd9b100f8");
// ...
});
await myFunc('myParam', 0);
Manually constructing and flushing the logger:
const { createMetricsLogger, Unit, StorageResolution } = require("aws-embedded-metrics");
const myFunc = async () => {
const metrics = createMetricsLogger();
metrics.putDimensions({ Service: "Aggregator" });
metrics.putMetric("ProcessingLatency", 100, Unit.Milliseconds, StorageResolution.Standard);
metrics.putMetric("Memory.HeapUsed", 1600424.0, Unit.Bytes, StorageResolution.High);
metrics.setProperty("RequestId", "422b1569-16f6-4a03-b8f0-fe3fd9b100f8");
// ...
await metrics.flush();
};
await myFunc();
If you are running on Lambda, export your function like so:
const { metricScope } = require("aws-embedded-metrics");
const myFunc = metricScope(metrics =>
async () => {
// ...
});
exports.handler = myFunc;
The MetricLogger
is the interface you will use to publish embedded metrics.
Adds a new metric to the current logger context. Multiple metrics using the same key will be appended to an array of values. Multiple metrics cannot have same key and different storage resolution. The Embedded Metric Format supports a maximum of 100 values per key. If more metric values are added than are supported by the format, the logger will be flushed to allow for new metric values to be captured.
Requirements:
InvalidMetricError
will be thrown. See MetricDatum for valid values.An OPTIONAL value representing the storage resolution for the corresponding metric. Setting this to High
specifies this metric as a high-resolution metric, so that CloudWatch stores the metric with sub-minute resolution down to one second. Setting this to Standard
specifies this metric as a standard-resolution metric, which CloudWatch stores at 1-minute resolution. If a value is not provided, then a default value of Standard
is assumed. See Cloud Watch High-Resolution metrics
Examples:
// Standard Resolution example
putMetric("Latency", 200, Unit.Milliseconds)
putMetric("Latency", 201, Unit.Milliseconds, StorageResolution.Standard)
// High Resolution example
putMetric("Memory.HeapUsed", 1600424.0, Unit.Bytes, StorageResolution.High);
Adds or updates the value for a given property on this context. This value is not submitted to CloudWatch Metrics but is searchable by CloudWatch Logs Insights. This is useful for contextual and potentially high-cardinality data that is not appropriate for CloudWatch Metrics dimensions.
Requirements:
Examples:
setProperty("RequestId", "422b1569-16f6-4a03-b8f0-fe3fd9b100f8")
setProperty("InstanceId", "i-1234567890")
setProperty("Device", {
Id: "61270781-c6ac-46f1-baf7-22c808af8162",
Name: "Transducer",
Model: "PT-1234"
});
Adds a new set of dimensions that will be associated to all metric values.
WARNING: Every distinct value will result in a new CloudWatch Metric.
If the cardinality of a particular value is expected to be high, you should consider
using setProperty
instead.
Requirements:
InvalidDimensionError
or DimensionSetExceededError
will be thrown. See Dimensions for valid values.Examples:
putDimensions({ Operation: "Aggregator" })
putDimensions({ Operation: "Aggregator", DeviceType: "Actuator" })
Explicitly override all dimensions. This will remove the default dimensions unless the useDefault
parameter is set to true
(defaults to false).
WARNING: Every distinct value will result in a new CloudWatch Metric.
If the cardinality of a particular value is expected to be high, you should consider
using setProperty
instead.
Requirements:
InvalidDimensionError
or DimensionSetExceededError
will be thrown. See Dimensions for valid values.Examples:
// Overwrites custom dimensions - keeps default dimensions
setDimensions({Operation: "Aggregator"}, true)
// Overwrites custom dimensions - removes default dimensions
setDimensions([
{ Operation: "Aggregator" },
{ Operation: "Aggregator", DeviceType: "Actuator" }
])
Explicitly clear all custom dimensions. Set useDefault
to true
to keep the default dimensions.
Example:
resetDimensions(false) // this will clear all custom dimensions as well as disable default dimensions
Sets the CloudWatch namespace that extracted metrics should be published to. If not set, a default value of aws-embedded-metrics will be used.
Requirements:
InvalidNamespaceError
will be thrown. See Namespace for valid values.Example:
setNamespace("MyApplication");
Sets the CloudWatch timestamp that extracted metrics are associated with. If not set a default value of new Date()
will be used.
If set for a given MetricsLogger
, timestamp will be preserved across calls to flush().
Requirements:
InvalidTimestampError
will be thrown.Examples:
setTimestamp(new Date())
setTimestamp(new Date().getTime())
Flushes the current MetricsContext to the configured sink and resets all properties and metric values. The namespace and default dimensions will be preserved across flushes. Custom dimensions are preserved by default, but this behavior can be changed by setting logger.flushPreserveDimensions = false
. Timestamp will be preserved if set explicitly via setTimestamp()
.
Examples:
logger.flush() // custom and default dimensions will be preserved after each flush
logger.flushPreserveDimensions = false
logger.flush() // only default dimensions will be preserved after flush()
logger.flushPreserveDimensions = false
logger.resetDimensions(false)
logger.flush() // default dimensions are disabled - no dimensions will be preserved after flush()
All configuration values can be set using environment variables with the prefix (AWS_EMF_
). Configuration should be performed as close to application start up as possible.
ServiceName: Overrides the name of the service. For services where the name cannot be inferred (e.g. Java process running on EC2), a default value of Unknown will be used if not explicitly set.
Requirements:
Example:
// in process
const { Configuration } = require("aws-embedded-metrics");
Configuration.serviceName = "MyApp";
// environment
AWS_EMF_SERVICE_NAME=MyApp
ServiceType: Overrides the type of the service. For services where the type cannot be inferred (e.g. Java process running on EC2), a default value of Unknown will be used if not explicitly set.
Requirements:
Example:
// in process
const { Configuration } = require("aws-embedded-metrics");
Configuration.serviceType = "NodeJSWebApp";
// environment
AWS_EMF_SERVICE_TYPE=NodeJSWebApp
LogGroupName: For agent-based platforms, you may optionally configure the destination log group that metrics should be delivered to. This value will be passed from the library to the agent in the Embedded Metric payload. If a LogGroup is not provided, the default value will be derived from the service name: -metrics
Requirements:
Example:
// in process
const { Configuration } = require("aws-embedded-metrics");
Configuration.logGroupName = "LogGroupName";
// environment
AWS_EMF_LOG_GROUP_NAME=LogGroupName
LogStreamName: For agent-based platforms, you may optionally configure the destination log stream that metrics should be delivered to. This value will be passed from the library to the agent in the Embedded Metric payload. If a LogGroup is not provided, the default value will be derived by the agent (this will likely be the hostname).
Requirements:
Example:
// in process
const { Configuration } = require("aws-embedded-metrics");
Configuration.logStreamName = "LogStreamName";
// environment
AWS_EMF_LOG_STREAM_NAME=LogStreamName
AgentEndpoint: For agent-based platforms, you may optionally configure the endpoint to reach the agent on.
Example:
// in process
const { Configuration } = require("aws-embedded-metrics");
Configuration.agentEndpoint = "udp://127.0.0.1:1000";
// environment
AWS_EMF_AGENT_ENDPOINT="udp://127.0.0.1:1000"
EnvironmentOverride: Short circuit auto-environment detection by explicitly defining how events should be sent. This is not supported through programatic access due to #43.
Valid values include:
Example:
AWS_EMF_ENVIRONMENT=Local
EnableDebugLogging: Enable debug logging for the library. If the library is not behaving as expected, you can set this to true to log to console.
Example:
// in process
const { Configuration } = require("aws-embedded-metrics");
Configuration.debuggingLoggingEnabled = true;
// environment
AWS_EMF_ENABLE_DEBUG_LOGGING=true
Namespace: Sets the CloudWatch namespace that extracted metrics should be published to. If not set, a default value of aws-embedded-metrics will be used.
Requirements:
Example:
// in process
const { Configuration } = require("aws-embedded-metrics");
Configuration.namespace = "Namespace";
// environment
AWS_EMF_NAMESPACE=Namespace
Check out the examples directory to get started.
Check out the unit test examples directory to get started. Here we provide a few examples to help you write tests against code that depends on this package.
This project uses Volta to pin the currently supported version of node.
npm i && npm run build
If you are running the CW agent locally, you can test the workflow:
npm i && npm link
cd examples/agent && npm link aws-embedded-metrics
After linking you'll need to rebuild any changes:
npm run build
We have 2 different types of tests:
npm test
# or
npm run watch
export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export AWS_REGION=us-west-2
npm run integ
We use Prettier for auto-formatting. You should install the plugin for your editor-of-choice and enabled format-on-save.
This project is licensed under the Apache-2.0 License.
FAQs
AWS Embedded Metrics Client Library
The npm package aws-embedded-metrics receives a total of 27,118 weekly downloads. As such, aws-embedded-metrics popularity was classified as popular.
We found that aws-embedded-metrics demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PEP 770 proposes adding SBOM support to Python packages to improve transparency and catch hidden non-Python dependencies that security tools often miss.
Security News
Socket CEO Feross Aboukhadijeh discusses open source security challenges, including zero-day attacks and supply chain risks, on the Cyber Security Council podcast.
Security News
Research
Socket researchers uncover how threat actors weaponize Out-of-Band Application Security Testing (OAST) techniques across the npm, PyPI, and RubyGems ecosystems to exfiltrate sensitive data.