New Case Study:See how Anthropic automated 95% of dependency reviews with Socket.Learn More
Socket
Sign inDemoInstall
Socket

github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awscontainerinsightreceiver

Package Overview
Dependencies
Alerts
File Explorer
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awscontainerinsightreceiver

  • v0.119.0
  • Source
  • Go
  • Socket score

Version published
Created
Source

AWS Container Insights Receiver

Status
Stabilitybeta: metrics
Distributionscontrib
WarningsOther
IssuesOpen issues Closed issues
Code Owners@Aneurysm9, @pxaws

Overview

AWS Container Insights Receiver (awscontainerinsightreceiver) is an AWS specific receiver that supports CloudWatch Container Insights. CloudWatch Container Insights collect, aggregate, and summarize metrics and logs from your containerized applications and microservices. Data are collected as as performance log events using embedded metric format. From the EMF data, Amazon CloudWatch can create the aggregated CloudWatch metrics at the cluster, node, pod, task, and service level.

CloudWatch Container Insights has been supported by ECS Agent and CloudWatch Agent to collect infrastructure metrics for many resources such as such as CPU, memory, disk, and network. To migrate existing customers to use OpenTelemetry, AWS Container Insights Receiver (together with CloudWatch EMF Exporter) aims to support the same CloudWatch Container Insights experience for the following platforms:

  • Amazon ECS
  • Amazon EKS
  • Kubernetes platforms on Amazon EC2

Design of AWS Container Insights Receiver

See the design doc

Configuration

Example configuration:

receivers:
  awscontainerinsightreceiver:
    # all parameters are optional
    collection_interval: 60s
    container_orchestrator: eks
    add_service_as_attribute: true 
    prefer_full_pod_name: false 
    add_full_pod_name_metric_label: false 

There is no need to provide any parameters since they are all optional.

collection_interval (optional)

The interval at which metrics should be collected. The default is 60 second.

container_orchestrator (optional)

The type of container orchestration service, e.g. eks or ecs. The default is eks.

add_service_as_attribute (optional)

Whether to add the associated service name as attribute. The default is true

prefer_full_pod_name (optional)

The "PodName" attribute is set based on the name of the relevant controllers like Daemonset, Job, ReplicaSet, ReplicationController, ... If it can not be set that way and PrefFullPodName is true, the "PodName" attribute is set to the pod's own name. The default value is false.

add_full_pod_name_metric_label (optional)

The "FullPodName" attribute is the pod name including suffix. If false FullPodName label is not added. The default value is false

Sample configuration for Container Insights

This is a sample configuration for AWS Container Insights using the awscontainerinsightreceiver and awsemfexporter for an EKS cluster:

# create namespace
apiVersion: v1
kind: Namespace
metadata:
  name: aws-otel-eks
  labels:
    name: aws-otel-eks

---
# create cwagent service account and role binding
apiVersion: v1
kind: ServiceAccount
metadata:
  name: aws-otel-sa
  namespace: aws-otel-eks

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: aoc-agent-role
rules:
  - apiGroups: [""]
    resources: ["pods", "nodes", "endpoints"]
    verbs: ["list", "watch"]
  - apiGroups: ["apps"]
    resources: ["replicasets"]
    verbs: ["list", "watch"]
  - apiGroups: ["batch"]
    resources: ["jobs"]
    verbs: ["list", "watch"]
  - apiGroups: [""]
    resources: ["nodes/proxy"]
    verbs: ["get"]
  - apiGroups: [""]
    resources: ["nodes/stats", "configmaps", "events"]
    verbs: ["create", "get"]
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["otel-container-insight-clusterleader"]
    verbs: ["get","update"]

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: aoc-agent-role-binding
subjects:
  - kind: ServiceAccount
    name: aws-otel-sa
    namespace: aws-otel-eks
roleRef:
  kind: ClusterRole
  name: aoc-agent-role
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-agent-conf
  namespace: aws-otel-eks
  labels:
    app: opentelemetry
    component: otel-agent-conf
data:
  otel-agent-config: |
    extensions:
      health_check:

    receivers:
      awscontainerinsightreceiver:

    processors:
      batch/metrics:
        timeout: 60s

    exporters:
      awsemf:
        namespace: ContainerInsights
        log_group_name: '/aws/containerinsights/{ClusterName}/performance'
        log_stream_name: '{NodeName}'
        resource_to_telemetry_conversion:
          enabled: true
        dimension_rollup_option: NoDimensionRollup
        parse_json_encoded_attr_values: [Sources, kubernetes]
        metric_declarations:
          # node metrics
          - dimensions: [[NodeName, InstanceId, ClusterName]]
            metric_name_selectors:
              - node_cpu_utilization
              - node_memory_utilization
              - node_network_total_bytes
              - node_cpu_reserved_capacity
              - node_memory_reserved_capacity
              - node_number_of_running_pods
              - node_number_of_running_containers
          - dimensions: [[ClusterName]]
            metric_name_selectors:
              - node_cpu_utilization
              - node_memory_utilization
              - node_network_total_bytes
              - node_cpu_reserved_capacity
              - node_memory_reserved_capacity
              - node_number_of_running_pods
              - node_number_of_running_containers
              - node_cpu_usage_total
              - node_cpu_limit
              - node_memory_working_set
              - node_memory_limit

          # pod metrics
          - dimensions: [[PodName, Namespace, ClusterName], [Service, Namespace, ClusterName], [Namespace, ClusterName], [ClusterName]]
            metric_name_selectors:
              - pod_cpu_utilization
              - pod_memory_utilization
              - pod_network_rx_bytes
              - pod_network_tx_bytes
              - pod_cpu_utilization_over_pod_limit
              - pod_memory_utilization_over_pod_limit
          - dimensions: [[PodName, Namespace, ClusterName], [ClusterName]]
            metric_name_selectors:
              - pod_cpu_reserved_capacity
              - pod_memory_reserved_capacity
          - dimensions: [[PodName, Namespace, ClusterName]]
            metric_name_selectors:
              - pod_number_of_container_restarts

          # cluster metrics
          - dimensions: [[ClusterName]]
            metric_name_selectors:
              - cluster_node_count
              - cluster_failed_node_count

          # service metrics
          - dimensions: [[Service, Namespace, ClusterName], [ClusterName]]
            metric_name_selectors:
              - service_number_of_running_pods

          # node fs metrics
          - dimensions: [[NodeName, InstanceId, ClusterName], [ClusterName]]
            metric_name_selectors:
              - node_filesystem_utilization

          # namespace metrics
          - dimensions: [[Namespace, ClusterName], [ClusterName]]
            metric_name_selectors:
              - namespace_number_of_running_pods


      debug:
        verbosity: detailed

    service:
      pipelines:
        metrics:
          receivers: [awscontainerinsightreceiver]
          processors: [batch/metrics]
          exporters: [awsemf]

      extensions: [health_check]

---
# create Daemonset
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: aws-otel-eks-ci
  namespace: aws-otel-eks
spec:
  selector:
    matchLabels:
      name: aws-otel-eks-ci
  template:
    metadata:
      labels:
        name: aws-otel-eks-ci
    spec:
      containers:
        - name: aws-otel-collector
          image: {collector-image-url}
          env:
            #- name: AWS_REGION
            #  value: "us-east-1"
            - name: K8S_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: HOST_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
            - name: HOST_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: K8S_NAMESPACE
              valueFrom:
                 fieldRef:
                   fieldPath: metadata.namespace
          imagePullPolicy: Always
          command:
            - "/awscollector"
            - "--config=/conf/otel-agent-config.yaml"
          volumeMounts:
            - name: rootfs
              mountPath: /rootfs
              readOnly: true
            - name: dockersock
              mountPath: /var/run/docker.sock
              readOnly: true
            - name: varlibdocker
              mountPath: /var/lib/docker
              readOnly: true
            - name: containerdsock
              mountPath: /run/containerd/containerd.sock
              readOnly: true
            - name: sys
              mountPath: /sys
              readOnly: true
            - name: devdisk
              mountPath: /dev/disk
              readOnly: true
            - name: otel-agent-config-vol
              mountPath: /conf
          resources:
            limits:
              cpu:  200m
              memory: 200Mi
            requests:
              cpu: 200m
              memory: 200Mi
      volumes:
        - configMap:
            name: otel-agent-conf
            items:
              - key: otel-agent-config
                path: otel-agent-config.yaml
          name: otel-agent-config-vol
        - name: rootfs
          hostPath:
            path: /
        - name: dockersock
          hostPath:
            path: /var/run/docker.sock
        - name: varlibdocker
          hostPath:
            path: /var/lib/docker
        - name: containerdsock
          hostPath:
            path: /run/containerd/containerd.sock
        - name: sys
          hostPath:
            path: /sys
        - name: devdisk
          hostPath:
            path: /dev/disk/
      serviceAccountName: aws-otel-sa

To deploy to an EKS cluster

kubectl apply -f config.yaml

Available Metrics and Resource Attributes

Cluster

MetricUnit
cluster_failed_node_countCount
cluster_node_countCount



Resource Attribute
ClusterName
NodeName
Type
Timestamp
Version
Sources





Cluster Namespace

MetricUnit
namespace_number_of_running_podsCount



Resource Attribute
ClusterName
NodeName
Namespace
Type
Timestamp
Version
Sources
kubernete





Cluster Service

MetricUnit
service_number_of_running_podsCount



Resource Attribute
ClusterName
NodeName
Namespace
Service
Type
Timestamp
Version
Sources
kubernete





Node

MetricUnit
node_cpu_limitMillicore
node_cpu_requestMillicore
node_cpu_reserved_capacityPercent
node_cpu_usage_systemMillicore
node_cpu_usage_totalMillicore
node_cpu_usage_userMillicore
node_cpu_utilizationPercent
node_memory_cacheBytes
node_memory_failcntCount
node_memory_hierarchical_pgfaultCount/Second
node_memory_hierarchical_pgmajfaultCount/Second
node_memory_limitBytes
node_memory_mapped_fileBytes
node_memory_max_usageBytes
node_memory_pgfaultCount/Second
node_memory_pgmajfaultCount/Second
node_memory_requestBytes
node_memory_reserved_capacityPercent
node_memory_rssBytes
node_memory_swapBytes
node_memory_usageBytes
node_memory_utilizationPercent
node_memory_working_setBytes
node_network_rx_bytesBytes/Second
node_network_rx_droppedCount/Second
node_network_rx_errorsCount/Second
node_network_rx_packetsCount/Second
node_network_total_bytesBytes/Second
node_network_tx_bytesBytes/Second
node_network_tx_droppedCount/Second
node_network_tx_errorsCount/Second
node_network_tx_packetsCount/Second
node_number_of_running_containersCount
node_number_of_running_podsCount



Resource Attribute
ClusterName
InstanceType
NodeName
Timestamp
Type
Version
Sources
kubernete





Node Disk IO

MetricUnit
node_diskio_io_serviced_asyncCount/Second
node_diskio_io_serviced_readCount/Second
node_diskio_io_serviced_syncCount/Second
node_diskio_io_serviced_totalCount/Second
node_diskio_io_serviced_writeCount/Second
node_diskio_io_service_bytes_asyncBytes/Second
node_diskio_io_service_bytes_readBytes/Second
node_diskio_io_service_bytes_syncBytes/Second
node_diskio_io_service_bytes_totalBytes/Second
node_diskio_io_service_bytes_writeBytes/Second



Resource Attribute
AutoScalingGroupName
ClusterName
InstanceId
InstanceType
NodeName
Timestamp
EBSVolumeId
device
Type
Version
Sources
kubernete




Node Filesystem

MetricUnit
node_filesystem_availableBytes
node_filesystem_capacityBytes
node_filesystem_inodesCount
node_filesystem_inodes_freeCount
node_filesystem_usageBytes
node_filesystem_utilizationPercent



Resource Attribute
AutoScalingGroupName
ClusterName
InstanceId
InstanceType
NodeName
Timestamp
EBSVolumeId
device
fstype
Type
Version
Sources
kubernete




Node Network

MetricUnit
node_interface_network_rx_bytesBytes/Second
node_interface_network_rx_droppedCount/Second
node_interface_network_rx_errorsCount/Second
node_interface_network_rx_packetsCount/Second
node_interface_network_total_bytesBytes/Second
node_interface_network_tx_bytesBytes/Second
node_interface_network_tx_droppedCount/Second
node_interface_network_tx_errorsCount/Second
node_interface_network_tx_packetsCount/Second



Resource Attribute
AutoScalingGroupName
ClusterName
InstanceId
InstanceType
NodeName
Timestamp
Type
Version
interface
Sources
kubernete




Pod

MetricUnit
pod_cpu_limitMillicore
pod_cpu_requestMillicore
pod_cpu_reserved_capacityPercent
pod_cpu_usage_systemMillicore
pod_cpu_usage_totalMillicore
pod_cpu_usage_userMillicore
pod_cpu_utilizationPercent
pod_cpu_utilization_over_pod_limitPercent
pod_memory_cacheBytes
pod_memory_failcntCount
pod_memory_hierarchical_pgfaultCount/Second
pod_memory_hierarchical_pgmajfaultCount/Second
pod_memory_limitBytes
pod_memory_mapped_fileBytes
pod_memory_max_usageBytes
pod_memory_pgfaultCount/Second
pod_memory_pgmajfaultCount/Second
pod_memory_requestBytes
pod_memory_reserved_capacityPercent
pod_memory_rssBytes
pod_memory_swapBytes
pod_memory_usageBytes
pod_memory_utilizationPercent
pod_memory_utilization_over_pod_limitPercent
pod_memory_working_setBytes
pod_network_rx_bytesBytes/Second
pod_network_rx_droppedCount/Second
pod_network_rx_errorsCount/Second
pod_network_rx_packetsCount/Second
pod_network_total_bytesBytes/Second
pod_network_tx_bytesBytes/Second
pod_network_tx_droppedCount/Second
pod_network_tx_errorsCount/Second
pod_network_tx_packetsCount/Second
pod_number_of_container_restartsCount
pod_number_of_containersCount
pod_number_of_running_containersCount
Resource Attribute
AutoScalingGroupName
ClusterName
InstanceId
InstanceType
K8sPodName
Namespace
NodeName
PodId
Timestamp
Type
Version
Sources
kubernete
pod_status



Pod Network

MetricUnit
pod_interface_network_rx_bytesBytes/Second
pod_interface_network_rx_droppedCount/Second
pod_interface_network_rx_errorsCount/Second
pod_interface_network_rx_packetsCount/Second
pod_interface_network_total_bytesBytes/Second
pod_interface_network_tx_bytesBytes/Second
pod_interface_network_tx_droppedCount/Second
pod_interface_network_tx_errorsCount/Second
pod_interface_network_tx_packetsCount/Second



Resource Attribute
AutoScalingGroupName
ClusterName
InstanceId
InstanceType
K8sPodName
Namespace
NodeName
PodId
Timestamp
Type
Version
interface
Sources
kubernete
pod_status




Container

MetricUnit
container_cpu_limitMillicore
container_cpu_requestMillicore
container_cpu_usage_systemMillicore
container_cpu_usage_totalMillicore
container_cpu_usage_userMillicore
container_cpu_utilizationPercent
container_memory_cacheBytes
container_memory_failcntCount
container_memory_hierarchical_pgfaultCount/Second
container_memory_hierarchical_pgmajfaultCount/Second
container_memory_limitBytes
container_memory_mapped_fileBytes
container_memory_max_usageBytes
container_memory_pgfaultCount/Second
container_memory_pgmajfaultCount/Second
container_memory_requestBytes
container_memory_rssBytes
container_memory_swapBytes
container_memory_usageBytes
container_memory_utilizationPercent
container_memory_working_setBytes
number_of_container_restartsCount



Resource Attribute
AutoScalingGroupName
ClusterName
ContainerId
ContainerName
InstanceId
InstanceType
K8sPodName
Namespace
NodeName
PodId
Timestamp
Type
Version
Sources
kubernetes
container_status
container_status_reason
container_last_termination_reason

The attribute container_status_reason is present only when container_status is in "Waiting" or "Terminated" State. The attribute container_last_termination_reason is present only when container_status is in "Terminated" State.

This is a sample configuration for AWS Container Insights using the awscontainerinsightreceiver and awsemfexporter for an ECS cluster to collect the instance level metrics:

receivers:
  awscontainerinsightreceiver:
    collection_interval: 10s
    container_orchestrator: ecs

processors:
  batch/metrics:
    timeout: 60s

exporters:
  awsemf:
    namespace: ContainerInsightsEC2Instance
    log_group_name: '/aws/ecs/containerinsights/{ClusterName}/performance'
    log_stream_name: 'instanceTelemetry/{ContainerInstanceId}'
    resource_to_telemetry_conversion:
      enabled: true
    dimension_rollup_option: NoDimensionRollup
    parse_json_encoded_attr_values: [Sources]
    metric_declarations:
      # instance metrics
      - dimensions: [ [ ContainerInstanceId, InstanceId, ClusterName] ]
        metric_name_selectors:
          - instance_cpu_utilization
          - instance_memory_utilization
          - instance_network_total_bytes
          - instance_cpu_reserved_capacity
          - instance_memory_reserved_capacity
          - instance_number_of_running_tasks
          - instance_filesystem_utilization
      - dimensions: [ [ClusterName] ]
        metric_name_selectors:
          - instance_cpu_utilization
          - instance_memory_utilization
          - instance_network_total_bytes
          - instance_cpu_reserved_capacity
          - instance_memory_reserved_capacity
          - instance_number_of_running_tasks
          - instance_cpu_usage_total
          - instance_cpu_limit
          - instance_memory_working_set
          - instance_memory_limit
  debug:
    verbosity: detailed
service:
  pipelines:
    metrics:
      receivers: [awscontainerinsightreceiver]
      processors: [batch/metrics]
      exporters: [awsemf,debug]

To deploy to an ECS cluster check this doc for details

Available Metrics and Resource Attributes

Instance

MetricUnit
instance_cpu_limitMillicore
instance_cpu_reserved_capacityPercent
instance_cpu_usage_systemMillicore
instance_cpu_usage_totalMillicore
instance_cpu_usage_userMillicore
instance_cpu_utilizationPercent
instance_memory_cacheBytes
instance_memory_failcntCount
instance_memory_hierarchical_pgfaultCount/Second
instance_memory_hierarchical_pgmajfaultCount/Second
instance_memory_limitBytes
instance_memory_mapped_fileBytes
instance_memory_max_usageBytes
instance_memory_pgfaultCount/Second
instance_memory_pgmajfaultCount/Second
instance_memory_reserved_capacityPercent
instance_memory_rssBytes
instance_memory_swapBytes
instance_memory_usageBytes
instance_memory_utilizationPercent
instance_memory_working_setBytes
instance_network_rx_bytesBytes/Second
instance_network_rx_droppedCount/Second
instance_network_rx_errorsCount/Second
instance_network_rx_packetsCount/Second
instance_network_total_bytesBytes/Second
instance_network_tx_bytesBytes/Second
instance_network_tx_droppedCount/Second
instance_network_tx_errorsCount/Second
instance_network_tx_packetsCount/Second
instance_number_of_running_tasksCount


Resource Attribute
ClusterName
InstanceType
AutoScalingGroupName
Timestamp
Type
Version
Sources
ContainerInstanceId
InstanceId





Instance Disk IO

MetricUnit
instance_diskio_io_serviced_asyncCount/Second
instance_diskio_io_serviced_readCount/Second
instance_diskio_io_serviced_syncCount/Second
instance_diskio_io_serviced_totalCount/Second
instance_diskio_io_serviced_writeCount/Second
instance_diskio_io_service_bytes_asyncBytes/Second
instance_diskio_io_service_bytes_readBytes/Second
instance_diskio_io_service_bytes_syncBytes/Second
instance_diskio_io_service_bytes_totalBytes/Second
instance_diskio_io_service_bytes_writeBytes/Second



Resource Attribute
ClusterName
InstanceType
AutoScalingGroupName
Timestamp
Type
Version
Sources
ContainerInstanceId
InstanceId
EBSVolumeId





Instance Filesystem

MetricUnit
instance_filesystem_availableBytes
instance_filesystem_capacityBytes
instance_filesystem_inodesCount
instance_filesystem_inodes_freeCount
instance_filesystem_usageBytes
instance_filesystem_utilizationPercent



Resource Attribute
ClusterName
InstanceType
AutoScalingGroupName
Timestamp
Type
Version
Sources
ContainerInstanceId
InstanceId
EBSVolumeId




Instance Network

MetricUnit
instance_interface_network_rx_bytesBytes/Second
instance_interface_network_rx_droppedCount/Second
instance_interface_network_rx_errorsCount/Second
instance_interface_network_rx_packetsCount/Second
instance_interface_network_total_bytesBytes/Second
instance_interface_network_tx_bytesBytes/Second
instance_interface_network_tx_droppedCount/Second
instance_interface_network_tx_errorsCount/Second
instance_interface_network_tx_packetsCount/Second



Resource Attribute
ClusterName
InstanceType
AutoScalingGroupName
Timestamp
Type
Version
Sources
ContainerInstanceId
InstanceId
EBSVolumeId




Warnings

Root permissions

When using this component, the collector process needs root permission to be able to read the content of the files located in the following locations:

  • /
  • /var/run/docker.sock
  • /var/lib/docker
  • /run/containerd/containerd.sock
  • /sys
  • /dev/disk

This requirement comes from the fact that this component is based on cAdvisor.

FAQs

Package last updated on 04 Feb 2025

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc