Testing AWS IAM policies
Introduction
AWS IAM policies are notouriously complex, it is too easy to add some unintended permissions and it is surprisingly difficult to identify these in heavily used AWS accounts.
Thankfully AWS has provided an IAM simulator that allows you to evaluate existing or new policies for its behavior. Which is very nice, but doing this manually is quite time consuming and it is unrealistic to test the entire environment for what you are trying to do.
However, in good AWS spirit the simulator has an API and this tool provides automation on top of it. It allows you to define the complete list of actions you want to evaluate against what resources, which allows you to run these tests on a regular basis or (better) integrate it in your CI/CD pipeline.
Testing approach
The testing leverages AWS' IAM simulator (api), that basically includes the same IAM evaluation logic that is applied when working in the console or using the cli. The beneits of this approach are:
- It takes all different levels of policies into account. Think about permission boundaries, service control policies and so on.
- It is an official service from AWS, so you can expect this to kept up to date over time.
- The actual actions are evaluated, but NOT executed. Hence no need for cleaning up resources after testing.
Quick testing
For convenience, you can use this tool to quickly test whether a user has a specific permission on a particular resource:
$ aws-iam-tester access -u the_user -a 'glue:DeleteTable'
Test:
Source: arn:aws:iam::208912673223:user/the_user
Action: glue:DeleteTable
Resource: *
Result: allowed
Matched statements:
Policy: admin_permissions
Type: IAM Policy
Start: L3:C17
End: L8:C6
Or to see who has access to a particular resource and action by omitting both user
and role
:
$ aws-iam-tester access -a "s3:PutObject"
..............
Summary:
| Source | Action | Resource | Decision | Policies |
|----------------------------------------------------------|--------------|------------|------------|---------------------|
| arn:aws:iam::XXXXXXXXXXXX:user/the_user | s3:PutObject | * | allowed | admins_permissions |
| arn:aws:iam::xxxxxxxxxxxx:role/the_role | s3:PutObject | * | allowed | AdministratorAccess |
Finding out who has access to a particular action/resource combination
It might be useful to check your highly senstive resources and find out who exactly can access these.
$ aws-iam-tester access -a "s3:PutObject" -R "arn:aws:s3:::my-strictly-confidential-data"
..............
Summary:
| Source | Action | Resource | Decision | Policies |
|----------------------------------------------------------|--------------|--------------------------------------------|------------|---------------------|
| arn:aws:iam::XXXXXXXXXXXX:user/the_user | s3:PutObject | arn:aws:s3:::my-strictly-confidential-data | allowed | admins_permissions |
| arn:aws:iam::xxxxxxxxxxxx:role/the_role | s3:PutObject | arn:aws:s3:::my-strictly-confidential-data | allowed | AdministratorAccess |
Account testing
However, the initial purpose of this tool is to check an entire account whether there are no users and/or roles having permissons which they should not have.
Configuration
In order to run, a configuration of the tests to run is required.
A sample configuration (with only one test) is shown, in various steps.
First there is a global section where you define settings which are applied to all tests (unless overruled, more on that later).
---
user_landing_account: 0123456789
global_exemptions:
- "^arn:aws:iam::(\\d{12}):user/(.*)(ADMIN|admin)(.*)$"
- "^arn:aws:iam::(\\d{12}):role/(.*)(ADMIN|admin)(.*)$"
- "^arn:aws:iam::(\\d{12}):role/AWSCloudFormationStackSetExecutionRole$"
Then you define a list of tests, each consisting at least of a set of:
- actions
- resources
- the expected result (should it fail or succeed)
tests:
- actions:
- "*:*"
- iam:*
- iam:AddUser*
- iam:Attach*
- iam:Create*
- iam:Delete*
- iam:Detach*
- iam:Pass*
- iam:Put*
- iam:Remove*
- iam:UpdateAccountPasswordPolicy
- sts:AssumeRole
- sts:AssumeRoleWithSAML
expected_result: fail
resources:
- "*"
Rather than using all users and roles (without exemptions) you can also limit your test to a particular set of users and roles.
The test below does that, including defining a custom context that specifies multi factor authentication is disabled when running the test. By default the context under which the simulations are run assumes MFA is enabled, but you can override that with the custom_context
element. For more information see the AWS documentation.
- actions:
- "*:*"
- iam:*
- iam:AddUser*
- iam:Attach*
- iam:Create*
- iam:Delete*
- iam:Detach*
- iam:Pass*
- iam:Put*
- iam:Remove*
- iam:UpdateAccountPasswordPolicy
- sts:AssumeRole
- sts:AssumeRoleWithSAML
expected_result: fail
resources:
- "*"
limit_to:
- "^arn:aws:iam::(\\d*):user/(.*)(ADMIN|admin)(.*)$"
- "^arn:aws:iam::(\\d*):role/(.*)(ADMIN|admin)(.*)$"
custom_context:
- context_key_name: aws:MultiFactorAuthPresent
context_key_values: false
context_key_type: boolean
Or if you want to do that for all tests you can use the global_limit_to
:
---
user_landing_account: 0123456789
global_limit_to:
- "^arn:aws:iam::(\\d{12}):user/(.*)(ENGINEER|engineer)(.*)$"
- "^arn:aws:iam::(\\d{12}):role/(.*)(SCIENTIST|scientist)(.*)$"
Below an example where an additional set of roles is exempt from testing:
- actions:
- redshift:GetClusterCredentials
- redshift:JoinGroup
- rds:Create*
- rds:Delete*
- rds:Modify*
- rds-db:connect
- s3:BypassGovernanceRetention
- s3:CreateBucket
- s3:DeleteBucket
- s3:DeleteBucketPolicy
- s3:PutBucketAcl
- s3:PutBucketPolicy
- s3:PutEncryptionConfiguration
- s3:ReplicateDelete
expected_result: fail
resources:
- "*"
exemptions: [
- "^arn:aws:iam::(\\d{12}):role/(.*)_worker$"
]
If you want to run positive tests (i.e. tests that you need to succeed rather than fail), these exemptions
don't work that well.
In that case you can limit your tests to a set of roles and users:
- actions:
- s3:PutObject
expected_result: succeed
resources:
- "arn:aws:s3:::my_bucket/xyz/*"
limit_to:
- "^arn:aws:iam::(\\d{12}):role/my_worker$"
Note that the exemptions are ignored when using a limit_to
list.
Using a dynamic account id in the resource arn
In case you need to specify the account id
in the resource arn, you can specificy this as follows:
- actions:
- "secretsmanager:GetSecretValue"
expected_result: succeed
resources:
- "arn:aws:secretsmanager:eu-central-1:{account_id}:secret:my-secret/*"
Using regular expressions for user and role matching
Regular expressions can be used when checking the users and/or roles that need to be included in the tests. Hence they are supported for the following elements in the config file:
- limit_to (the filtered list of users/roles to be used for a particular test)
- global_limit_to (the filtered list of users/roles to be used for all tests)
- exemptions (the filtered list of users/roles that should be excluded from a particular test)
- global_exemptions (the filtered list of users/roles that should be excluded from all tests)
For all other elements, regular expression matching is NOT supported!
How to use
Assuming you have define a config.yml in your local directory, then to run and write the outputs to the local ./results
directory:
aws-iam-tester --write-to-file
Using a specific config file:
aws-iam-tester --config-file my-config.yml
Using a specific output location:
aws-iam-tester --output-location /tmp
Or write to s3:
aws-iam-tester --output-location s3://my-bucket/my-prefix
Include only roles that can be assumed by human beings:
aws-iam-tester --no-include-system-roles
Note: including system roles does NOT include the aws service roles.
Or print debug output:
aws-iam-tester --debug
To run a limited number of evaluations (which helps speeding things up, and avoiding API throttling issues):
aws-iam-tester --number-of-runs 10
For more information, run aws-iam-tester --help
for more instructions.
Return codes
The tester returns the following return codes:
- 0 upon successful completion with NO findings
- 1 upon successful completion with findings
- 2 (or higher) on failures
Required permissions
Obviously the client has to run under an AWS security context that has sufficient permissions to query the IAM resources and run the simulator.
The following permissions are needed at the minimum:
- sts:GetCallerIdentity
- iam:ListAccountAliases
- iam:ListRoles
- iam:ListUsers
- iam:SimulatePrincipalPolicy
And if you want to write the output to an s3 location, then obviously you need write access (s3:PutObject
) to that particular location as well.
Unit testing
pytest
is being used for testing the various options.
As long as the aws-iam-tester
module is installed, you can run the tests.
pytest -vsx
# or, alternatively
poetry run pytest -vsx
After installing tox
, you can also simply run $ tox
.