Cloud Custodian, also known as c7n, is a rules engine for managing public cloud accounts and resources. It allows users to define policies to enable a well managed cloud infrastructure, that's both secure and cost optimized. It consolidates many of the adhoc scripts organizations have into a lightweight and flexible tool, with unified metrics and reporting.
Custodian can be used to manage AWS, Azure, and GCP environments by ensuring real time compliance to security policies (like encryption and access requirements), tag policies, and cost management via garbage collection of unused resources and off-hours resource management.
Custodian also supports running policies on infrastructure as code assets to provide feedback directly on developer workstations or within CI pipelines.
Custodian policies are written in simple YAML configuration files that enable users to specify policies on a resource type (EC2, ASG, Redshift, CosmosDB, PubSub Topic) and are constructed from a vocabulary of filters and actions.
It integrates with the cloud native serverless capabilities of each provider to provide for real time enforcement of policies with builtin provisioning. Or it can be run as a simple cron job on a server to execute against large existing fleets.
Cloud Custodian is a CNCF Incubating project, lead by a community of hundreds of contributors.
Custodian is published on pypi as a series of packages with the c7n
prefix, its also available as a docker image.
$ python3 -m venv custodian
$ source custodian/bin/activate
(custodian) $ pip install c7n
The first step to using Cloud Custodian (c7n) is writing a YAML file containing the policies that you want to run. Each policy specifies the resource type that the policy will run on, a set of filters which control resources will be affected by this policy, actions which the policy with take on the matched resources, and a mode which controls which how the policy will execute.
The best getting started guides are the cloud provider specific tutorials.
As a quick walk through, below are some sample policies for AWS resources.
policies:
- name: s3-cross-account
description: |
Checks S3 for buckets with cross-account access and
removes the cross-account access.
resource: aws.s3
region: us-east-1
filters:
- type: cross-account
actions:
- type: remove-statements
statement_ids: matched
- name: ec2-require-non-public-and-encrypted-volumes
resource: aws.ec2
description: |
Provision a lambda and cloud watch event target
that looks at all new instances and terminates those with
unencrypted volumes.
mode:
type: cloudtrail
role: CloudCustodian-QuickStart
events:
- RunInstances
filters:
- type: ebs
key: Encrypted
value: false
actions:
- terminate
- name: tag-compliance
resource: aws.ec2
description: |
Schedule a resource that does not meet tag compliance policies to be stopped in four days. Note a separate policy using the`marked-for-op` filter is required to actually stop the instances after four days.
filters:
- State.Name: running
- "tag:Environment": absent
- "tag:AppId": absent
- or:
- "tag:OwnerContact": absent
- "tag:DeptID": absent
actions:
- type: mark-for-op
op: stop
days: 4
You can validate, test, and run Cloud Custodian with the example policy with these commands:
# Validate the configuration (note this happens by default on run)
$ custodian validate policy.yml
# Dryrun on the policies (no actions executed) to see what resources
# match each policy.
$ custodian run --dryrun -s out policy.yml
# Run the policy
$ custodian run -s out policy.yml
You can run Cloud Custodian via Docker as well:
# Download the image
$ docker pull cloudcustodian/c7n
$ mkdir output
# Run the policy
#
# This will run the policy using only the environment variables for authentication
$ docker run -it \
-v $(pwd)/output:/home/custodian/output \
-v $(pwd)/policy.yml:/home/custodian/policy.yml \
--env-file <(env | grep "^AWS\|^AZURE\|^GOOGLE") \
cloudcustodian/c7n run -v -s /home/custodian/output /home/custodian/policy.yml
# Run the policy (using AWS's generated credentials from STS)
#
# NOTE: We mount the ``.aws/credentials`` and ``.aws/config`` directories to
# the docker container to support authentication to AWS using the same credentials
# credentials that are available to the local user if authenticating with STS.
$ docker run -it \
-v $(pwd)/output:/home/custodian/output \
-v $(pwd)/policy.yml:/home/custodian/policy.yml \
-v $(cd ~ && pwd)/.aws/credentials:/home/custodian/.aws/credentials \
-v $(cd ~ && pwd)/.aws/config:/home/custodian/.aws/config \
--env-file <(env | grep "^AWS") \
cloudcustodian/c7n run -v -s /home/custodian/output /home/custodian/policy.yml
The custodian cask tool is a go binary that provides a transparent front end to docker that mirors the regular custodian cli, but automatically takes care of mounting volumes.
Consult the documentation for additional information, or reach out on gitter.
For specific instructions for AWS, Azure, and GCP, visit the relevant getting started page.
cloudcustodian tagWe have a regular community meeting that is open to all users and developers of every skill level. Joining the mailing list will automatically send you a meeting invite. See the notes below for more technical information on joining the meeting.
The Custodian project also develops and maintains a suite of additional tools here https://github.com/cloud-custodian/cloud-custodian/tree/master/tools:
Org: Multi-account policy execution.
ShiftLeft: Shift Left ~ run policies against Infrastructure as Code assets like terraform.
PolicyStream: Git history as stream of logical policy changes.
Salactus: Scale out s3 scanning.
Mailer: A reference implementation of sending messages to users to notify them.
Trail Creator: Retroactive tagging of resources creators from CloudTrail
TrailDB: Cloudtrail indexing and time series generation for dashboarding.
LogExporter: Cloud watch log exporting to s3
Cask: Easy custodian exec via docker
Guardian: Automated multi-account Guard Duty setup
Omni SSM: EC2 Systems Manager Automation
Mugc: A utility used to clean up Cloud Custodian Lambda policies that are deployed in an AWS environment.
See https://cloudcustodian.io/docs/contribute.html
If you've found a security related issue, a vulnerability, or a potential vulnerability in Cloud Custodian please let the Cloud Custodian Security Team know with the details of the vulnerability. We'll send a confirmation email to acknowledge your report, and we'll send an additional email when we've identified the issue positively or negatively.
This project adheres to the CNCF Code of Conduct
By participating, you are expected to honor this code.
FAQs
Cloud Custodian - Policy Rules Engine
We found that c7n demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 4 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.