2

If I call the boto3 EC2 client's describe_instances function with no MaxResults parameter, will it return all instances in the initial call? There is a parameter that allows one to specify MaxResults, but it is not required. If I don't specify this MaxResults parameter, will the response contain all instances or will it still chunk them into groups using the NextToken of the response?

The documentation says

"Describes the specified instances or all of AWS account's instances...If you do not specify instance IDs, Amazon EC2 returns information for all relevant instances."

But it is not clear whether I still need to expect that things could be returned in chunks if my account has a lot of instances. The MaxResults parameter can be set to "between 5 and 1000," which implies 1000 may be the default MaxResults.

Shawn
  • 8,374
  • 5
  • 37
  • 60

3 Answers3

3

If you do not specify MaxResults, then the server-side API will limit the response to either a maximum number of results/items (for example 1000) or a maximum size of response payload (e.g. 256 MB). Which it does is not typically documented, and potentially varies from API call to API call and from service to service.

If NextToken is present in the response and is not NULL, then you should re-issue the API call, with the NextToken, to get the next 'page' of results. Rinse and repeat until you have all results.

If you know you have only a handful of EC2 instances (say < 100), most programmers don't typically check the response's NextToken. They probably should, but they don't.

Note that the above relates to the boto3 Client interface. You can also use the describe-instances paginator.

If you are purely interested in EC2 instances within a given VPC, then you can use the VPC's instances collection. This is part of boto3's Resource-level interface. The instances are lazily-loaded and you don't need to paginate or mess with next tokens. See differences between Client and Resource.

jarmod
  • 71,565
  • 16
  • 115
  • 122
1

Modified

Let us assume that we call the describe_instances() and didn't set the value of MaxResults.

Then, the response will contain the list of instances. There can be NextToken or not. If NextToken exists, the response is showing only some part of all instances. If NextToken is not present, then the response shows all instances.

Not setting the MaxResults does not mean that the response will show all instances.

Original

Once you receive the response as a result of describe_instances() without NextToken, the result shows all instances even you didn't set the MaxResults. You only need to care about the response for describe_instances().

Or use the pagenator to get all result without NextToken. Here is my sample code for snapshot.

import boto3

boto3 = boto3.session.Session(region_name='ap-northeast-2')
ec2 = boto3.client('ec2')

page_iterator = ec2.get_paginator('describe_snapshots').paginate()

for page in page_iterator:
    for snapshot in page['Snapshots']:
        print(snapshot['SnapshotId'], snapshot['StartTime'])

This will print all snapshot id and starttime.

Shawn
  • 8,374
  • 5
  • 37
  • 60
Lamanus
  • 12,898
  • 4
  • 21
  • 47
  • I understand, but I am asking if the initial response will contain all results--essentially, if I even need to bother looking for NextToken if I don't specify MaxResults – Shawn Aug 29 '19 at 15:07
  • If you dont want to care about next toke then use pagenator as introduced by the other answer. Then you will always get the total result not the partial. – Lamanus Aug 29 '19 at 15:09
  • I appreciate the input, but you aren't answering my question. I am familiar with how paginators work. I am not asking about a call to the "get_paginator" function; rather, I am asking about a call to the "describe_instances" function. I'm trying to understand what the default MaxResults is for the "describe_instances" function. If I don't specify that parameter, will my results be limited, and if so, to how many? In some _other_ API call documentation it says things like "Returns some or all (up to 1000)" but _not_ for the describe_instances documentation I linked in my question. – Shawn Aug 29 '19 at 19:19
  • It is the same with other boto3 APIs. If you didn't specify the MaxResults, the result will be paginated with NextToken. There is no way to get the whole instances without pagination. It will be assumed for some result ans will give you a NextToken. Sorry for mis-understand your question. – Lamanus Aug 30 '19 at 03:39
0

Check the below 2 options to call describe instances:

  • simple direct API call like "describe_instances" which has NextToken argument which means you can use this token as a starting point when you query next time. If you have less number of instances maybe in a single call it would return all instances and in that case you wont see NextToken value.
  • Using paginator Command Reference Here once you have a paginator.paginate() object, you can use for loop and it will return all instances. In this way we don't have to worry about MaxItems or NextToken. Simple example that illustrates how to use a paginator

I would recommend using paginators whenever possible.

jestadi
  • 96
  • 6