How to iterate through multiple AWS accounts?

Question

Consider an AWS organization with 50+ accounts. Each account has a role created that allows read-only access to the EC2 service - named "EC2ReadAccess" - and a trust relationship with the master(/management) account. A single IAM user in the master account has a policy applied to allow it to assume the role in every single account.

I would like to be able to iterate through all the accounts in order to retrieve all the EC2 instances, using the unique IAM user. I know that running a CLI command against all the accounts at once is out of the question. Leaving aside cycling through the regions in each account, which has been discussed extensively, is there an elegant solution for doing this?

One hack that comes to mind is to programmatically build the credentials file so that it contains a profile for each account. Each profile, in turn, will be "linked" to the profile of the IAM user (as described here), and have the account id within the role updated for each entry. An example below:

[user1]
aws_access_key_id=<key_id>
aws_secret_access_key=<secret_key>

[marketing]
role_arn = arn:aws:iam::123456789012:role/EC2ReadAccess
source_profile = user1
[dev]
role_arn = arn:aws:iam::234567890123:role/EC2ReadAccess
source_profile = user1
...
[prod]
role_arn = arn:aws:iam::345678901234:role/EC2ReadAccess
source_profile = user1

Running the CLI command - in this case aws ec2 describe-instances - can be accompanied by the --profile parameter in order to cycle through all the profiles present in the credentials file. The region can be cycled through within another loop. So a list of commands to be issued, generated programmatically beforehand, could look something like:

aws ec2 describe-instances --profile marketing --region us-east-1
aws ec2 describe-instances --profile marketing --region us-east-2
...
aws ec2 describe-instances --profile marketing --region sa-east-1
aws ec2 describe-instances --profile dev --region us-east-1
aws ec2 describe-instances --profile dev --region us-east-2
...
aws ec2 describe-instances --profile dev --region sa-east-1
...
aws ec2 describe-instances --profile prod --region us-east-1
aws ec2 describe-instances --profile prod --region us-east-2
...
aws ec2 describe-instances --profile prod --region sa-east-1

Is there a better way of doing this? Am I missing something obvious here?

Answer 1

Don't use the CLI to do this. If you're able to "programmatically build the credentials file", then you should be able to write a program that does the following for every child account:

1. Assumes the administrator role for that account.
2. Invokes the DescribeInstances API call.
3. Do whatever you want with the information

You don't indicate what programming language(s) you're familiar with, but here's some Python that I use to create an SDK client with the standard "OrganizationalAccountAccessRole" in a child account:

def create_boto_resource(account, region, resource_type):
    creds = assumeRole(account)
    return boto3.resource(resource_type,
                          region_name=region,
                          aws_access_key_id=creds['AccessKeyId'],
                          aws_secret_access_key=creds['SecretAccessKey'],
                          aws_session_token=creds['SessionToken'])



def assumeRole(account, role_name='OrganizationAccountAccessRole', duration=900):
    stsClient = boto3.client('sts')
    request = {}
    request['RoleArn']         = f"arn:aws:iam::{account}:role/{role_name}"
    request['RoleSessionName'] = 'DESCRIPTIVE_TEXT_HERE'
    request['DurationSeconds'] = duration
    return stsClient.assume_role(**request)['Credentials']

This code actually creates a Boto resource object, which is a simplified API that's supported for EC2 and a few other services. Here's and example of using the resource API to create an instance iterator.

Answer 2

Another way to attack this kind of problem might be via Systems Manager Inventory and Querying inventory data from multiple Regions and accounts.