Getting batches from a list jpw

Question

I have a function that should get parts from a list and than move in the list to get the next part or part:

def batch(iterable, n=1):
   l = len(iterable)
   for ndx in range(0, l, n):
      yield iterable[ndx:min(ndx + n, l)]

I've tried to use this function as followed :

.............................
index =0; 
values.clear()

for i in batch(my_data, 40):
   values.append(i)
   index = index +1 
   print('index', index )

UPDATE

what I'm seeing is that index value reaches 58 or 20, while I'm expecting 40

thanks in advance !

"I expect to 40 as a value but I get 58 and sometimes 20 I just get it." Sorry, I don't understand that sentence. Could you post some actual input and output? — tobias_k, Apr 24 '17 at 15:06
Also, no need for `min`. `iterable[ndx:ndx + n]` should work in any case (if `iterable` supports slicing, of course) — tobias_k, Apr 24 '17 at 15:07
there is probably something in `itertools` for this task already — Ma0, Apr 24 '17 at 15:08
Not in `itertools`, AFAIK, but some related question on SO, e.g. [this](http://stackoverflow.com/q/24527006/1639625) or [this](http://stackoverflow.com/q/434287/1639625) — tobias_k, Apr 24 '17 at 15:09
Why do you expect `index` to reach `40`? Do you want to split the iterable into `n` chunks, or into chunks with size `n`? Your code does the latter. — tobias_k, Apr 24 '17 at 15:13
well AFAIK batch function will reader 40 element from the list, so if I'm increment the index during this op, it'll reach the value of 4 0 — Engine, Apr 24 '17 at 15:15
@Ev.Kounis Why? OP is not iterating _one_ batch, but _all_ the batches. The only reason for `index` to end up at 40 would be if the original data has around 1600 entries. — tobias_k, Apr 24 '17 at 15:17
http://stackoverflow.com/questions/8290397/how-to-split-an-iterable-in-constant-size-chunks — Engine, Apr 24 '17 at 15:18

score 1 · Accepted Answer · answered Apr 24 '17 at 15:22

1

There seems to be some confusion as to what the code does. When called with n=40, it does not create 40 batches, but batches that are each 40 elements long. Thus, if you iterate the batches like you do in your loop and increase index in each iteration, it does not have to end up at 40.

Take a look at this smaller example:

lst = list(range(20))
for b in batch(lst, 7):
    print(b)

In the output, you can see that there are not 7 but just 3 batches.

[0, 1, 2, 3, 4, 5, 6]
[7, 8, 9, 10, 11, 12, 13]
[14, 15, 16, 17, 18, 19]

answered Apr 24 '17 at 15:22

tobias_k

81,265
12
120
179

should this not print then `7` then `14` and then `20` with the `print(index)`? – Ma0 Apr 24 '17 at 15:26
@Ev.Kounis Why? OP is doing `index = index + 1`, not `index = index + len(i)` in the loop. Thus, `index` is just the number of iterations of the loop, i.e. the number of batches. – tobias_k Apr 24 '17 at 15:30
@ Tobias thanks for the clarification, I've got it now – Engine Apr 25 '17 at 07:26

Getting batches from a list jpw

1 Answers1