With Boto3, how can I get volumes that are smaller than Y in a list of EC2 Instances?

Question

I have ~400 redis servers that need to have 60gb EBS vols attached. Some older nodes are going to be less than 60gb.

My approach was going to be iterate over each instance, find the volume, if volume is less than 60GB then return instance_id but it doesn't seem to work.

redis = []
def has_small_vols(instlist):
    for i in instlist:
        instance = ec2.Instance(str(i))
        instid = i.instance_id
        vols = instance.volumes.all()
        for volume in vols:
            if volume.size < 60:
                redis.append(instid)

but something is wrong in the for volume in vols loop and I'm unsure why. I got that idea from here

I have tried boto3.resource('ec2') and I am unsure I need to use boto3.client('ec2') as well.

Creating the list of instances is not the problem:

import boto3

ec2 = boto3.resource('ec2')

def get_redis_nodes():
    filters = [{'Name':'tag:Service', 'Values':['redis']}]
    filt = [{'Name':'tag:Environment', 'Values':['production*']}]
    instlist = list(ec2.instances.filter(Filters=filters).filter(Filters=filt).instance_id)
    return instlist

What I expected is that the code goes through filtered instances, grab EBS volumes, finds EBS volumes that fit the IF and then append to a list.

But, if I try and print through the iterations I don't get the volume size:

     for i in instlist:
         instance = ec2.Instance(str(i))
         vols = instance.volumes.all()
         print(i.instance_id)
         print(vols)
         for volume in vols:
             print(volume.size)
>>> get_redis_info()
i-a689ba6efa
ec2.Instance.volumesCollection(ec2.Instance(id="ec2.Instance(id='i-a689ba6efa')"), ec2.Volume)
i-f4b8212aev5748d
ec2.Instance.volumesCollection(ec2.Instance(id="ec2.Instance(id='i-f4b8212aev5748d')"), ec2.Volume)
i-0Ad235afh3a1d0f4
ec2.Instance.volumesCollection(ec2.Instance(id="ec2.Instance(id='i-0Ad235afh3a1d0f4')"), ec2.Volume)

jarmod · Accepted Answer · 2019-09-03T19:33:58.863

Your code to filter and retrieve instances has minor bugs in it.

Note that the ec2.instances.filter() method on the EC2 resource API returns EC2 instances, as in objects of type ec2.Instance, and there's no need to reduce that to a list of instance IDs and then later, in your has_small_vols() method, convert the instance IDs back to objects of type ec2.Instance. You can simply use the list of ec2.Instance objects throughout, converting to a list of instance IDs at the end, if needed.

Try the following:

import boto3

ec2 = boto3.resource('ec2')

def has_small_vols(instance):
    vols = instance.volumes.all()
    for volume in vols:
        if volume.size < 60:
            return True
    return False

def get_redis_nodes():
    filters = [
        {'Name':'tag:Service', 'Values':['redis']},
        {'Name':'tag:Environment', 'Values':['production*']}
    ]
    return ec2.instances.filter(Filters=filters)

redis_instances = get_redis_nodes()
redis_instances_small = [i for i in redis_instances if has_small_vols(i)]
print(redis_instances_small)

redis_small_ids = [i.id for i in redis_instances_small]
print(redis_small_ids)

Note that the client and resource APIs in boto3 are quite different.

I love this answer - thank you very much. I had realized my error in my has_small_vols() converting my instance ids back to objects right as you answered. But, this is perfect and the one thing I love about this is how clean this code is. Thank you for helping me! — E.Z., Sep 03 '19 at 19:46

With Boto3, how can I get volumes that are smaller than Y in a list of EC2 Instances?

1 Answers1