Grabbing values of an array in sets of 100

Question

In the code below, ids is an array which contains the steam64 ids of all users in your friendslist. Now according to the steam web api documentation, GetPlayerSummaries only takes a list of 100 comma separated steam64 ids. Some users have more than 100 friends, and instead of running a for loop 200 times that each time calls the API, I want to take array in sets of 100 steam ids. What would be the most efficient way to do this (in terms of speed)?

I know that I can do ids[0:100] to grab the first 100 elements of an array, but how I accomplish doing this for a friendlist of say 230 users?

def getDescriptions(ids):
    sids = ','.join(map(str, ids)) 
    r = requests.get('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key='+API_KEY+'&steamids=' + sids)
    data = r.json();
...

possible duplicate of [Iterate an iterator by chunks (of n) in Python?](http://stackoverflow.com/questions/8991506/iterate-an-iterator-by-chunks-of-n-in-python) — a p, Oct 27 '14 at 17:04

score 0 · Accepted Answer · edited May 23 '17 at 12:09

Utilizing the code from this answer, you are able to break this into groups of 100 (or less for the last loop) of friends.

def chunkit(lst, n):
    newn = int(len(lst)/n)
    for i in xrange(0, n-1):
        yield lst[i*newn:i*newn+newn]
    yield lst[n*newn-newn:]


def getDescriptions(ids):
    friends = chunkit(ids, 3)
    while (True):
        try:
            fids = friends.next()
            sids = ','.join(map(str, fids)) 
            r = requests.get('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key='+API_KEY+'&steamids=' + sids)
            data = r.json()
            # Do something with the data variable
        except StopIteration:
            break

This will create iterators broken into 3 (second parameter to chunkit) groups. I chose 3, because the base size of the friends list is 250. You can get more (rules from this post), but it is a safe place to start. You can fine tune that value as you need.

Utilizing this method, your data value will be overwritten each loop. Make sure you do something with it at the place indicated.

would it be easier to use this code `def chunksiter(l, chunks): i,j,n = 0,0,0 rl = [] while n < len(l)/chunks: rl.append(l[i:j+chunks]) i+=chunks j+=j+chunks n+=1 return iter(rl)` and use this code with `chunks = 100` — Vishwa Iyer, Oct 27 '14 at 17:30

Anzel · Answer 2 · 2014-10-27T18:16:47.793

-1

I have an easy alternative, just reduce your list size on each while/loop until exhaustion:

def getDescriptions(ids):
    sids = ','.join(map(str, ids))
    sids_queue = sids.split(',')
    data = []
    while len(sids_queue) != 0:
        r = requests.get('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key='+ \
                            API_KEY+'&steamids=' + ','.join(sids_queue[:100])
        data.append(r.json) # r.json without (), by the way
        # then skip [0:100] and reassign to sids_queue, you get the idea
        sids_queue = sids_queue[101:]

edited Oct 27 '14 at 18:16

answered Oct 27 '14 at 17:31

Anzel

19,825
5
51
52

@Andy, The document you show probably is wrong, as I try their example and it gives me a nice `TypeError: 'dict' object is not callable` – Anzel Oct 27 '14 at 17:39
The document is not incorrect. Your code results in this (for one iteration through your loop): `[>]`. Additionally, your `.get` throws this error: `TypeError: cannot concatenate 'str' and 'list' objects` – Andy Oct 27 '14 at 17:45
1

@Andy thanks for spotting it. I edited and put `join` to the list now – Anzel Oct 27 '14 at 18:17

Grabbing values of an array in sets of 100

2 Answers2