How to retrieve all rows from Azure Table Storage

Question

I'm querying Azure Table storage with the following code:

queried_entities = table_service.query_entities('myTable', filter="RowKey gt '2017-06-20T15:33:00' and PartitionKey eq 'myKey'"

This however only returns the first 1000 entries. Is there a way to page through the rest or download all rows that match the query?

Didn't realize that I had already answered a similar question some time back :). Closing this as duplicate of that. I hope that's OK. — Gaurav Mantri, Jun 23 '17 at 08:05
Thanks. The .NET example is actually even more useful. This works. — s952163, Jun 23 '17 at 08:14

score 1 · Answer 1 · answered Jun 23 '17 at 07:19

In a single request to Azure Table Service, a maximum of 1000 entities will be returned (it could be anywhere from 0 to 1000). If there are more entities available, then Table Service will return you an opaque continuation token that you would need to use to fetch next set of entities.

What you need to do is fetch entities in an endless loop. In each iteration, you will pass your query and continuation token returned from previous request (you would pass null as continuation token for the 1st iteration). In result, you will get entities and optionally a continuation token (if more entities are available). You will continue this loop till the time you don't get back any continuation token.

Unfortunately I don't have Python environment setup, hence I can't show the code. However, I answered a similar question where I included code using .Net SDK. You can take a look at the code here: How to query all rows in windows azure table storage?. This should give you an idea about how you would accomplish the same in Python.

How to retrieve all rows from Azure Table Storage

1 Answers1