Python 2 --> 3: object of type 'zip' has no len()

Question

I'm following a tutorial on neural nets1

It's in Python 2.7. I'm using 3.4. This is the line that troubles me:

if test_data: n_test = len(test_data)

I get: TypeError: object of type 'zip' has no len().

Is there a way to rewrite it so that it works in 3.4?

related: [Length of a finite generator](http://stackoverflow.com/q/18014437/4279) — jfs, Jun 23 '15 at 20:18
related: [Getting number of elements in an iterator in Python](http://stackoverflow.com/q/3345785/4279) — jfs, Jun 23 '15 at 20:20
If you have access to the two iterables that have been zipped, just compute the length of both and take whichever is smaller. The length of the zip is not going to be any different from that. — Akshat Mahajan, Mar 30 '16 at 02:27
That appears to have fixed it, although I get a second error in regards to `mini_batches = [training_data[k:k+mini_batch_size] for k in range(0, n, mini_batch_size)]` the error reads `File "/home/lukasz/Documents/Machine Learning/network.py", line 66, in for k in range(0, n, mini_batch_size)] TypeError: 'zip' object is not subscriptable ` w — Lukasz, Jun 27 '16 at 04:28
same problem, the program is trying to do something like `zip(...)[i]`. Best would be to define a new variable `new_var = list(zip(...))` and replacing `zip(...)` with `new_var`. — Julien, Jun 27 '16 at 04:35

score 64 · Answer 1 · answered Mar 30 '16 at 02:23

64

A bit late now to answer, but in case anyone else stumbles on it: for that same neural net example tutorial, it turned out I had to wrap the 3 zip calls in the mnist_loader with a list(zip(...)) construct:

training_data = list(zip(training_inputs, training_results))
(...)
validation_data = list(zip(validation_inputs, va_d[1]))
(...)
test_data = list(zip(test_inputs, te_d[1]))

And then it worked.

answered Mar 30 '16 at 02:23

Nicolas Turgeon

649
5
2

4

late but welcome for those who arrive even later, ty – Haboryme Sep 26 '16 at 16:48
1

All three in `load_data_wrapper()`. – EmmanuelMess Jul 03 '17 at 01:48
3

they are talking about this tutorial: http://neuralnetworksanddeeplearning.com/chap1.html – Slawa Nov 29 '17 at 12:08

jfs · Accepted Answer · 2015-06-23T21:06:30.957

38

If you know that the iterator is finite:

#NOTE: `sum()` consumes the iterator
n_test = sum(1 for _ in test_data) # find len(iterator)

Or if you know that test_data is always small and a profiler says that the code is the bottleneck in your application then here's code that might be more efficient for small n_test:

test_data = list(test_data)
n_test = len(test_data)

Unfortunately, operator.length_hint() (Python 3.4+) returns zero for a zip() object. See PEP 0424 -- A method for exposing a length hint.

edited Jun 23 '15 at 21:06

answered Jun 23 '15 at 20:17

jfs

399,953
195
994
1,670

3

You should do `test_data = list(test_data)` before the `if` check, otherwise `if test_data` will always be true, even for an empty `zip`. Also might be worth pointing out that `sum(...)` will consume the `zip` – tobias_k Jun 23 '15 at 21:02
I see you point... in fact, the `if` check would not be needed if it were just an empty list (then `n_test` would just be 0). Still, might be worth pointing out, just in case. – tobias_k Jun 23 '15 at 21:12
Working through your suggestions now, thanks a lot. It's someone else's code and I admit it's stretching my ability. Cheers! – Ada Stra Jun 24 '15 at 05:02

score 13 · Answer 3 · answered Jun 23 '15 at 19:24

13

Force the zip() to evaluate.

foo = list(zip(...))

answered Jun 23 '15 at 19:24

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

score 3 · Answer 4 · answered Jun 27 '16 at 04:10

3

Some Info

This is because in the Python 3.x, zip returns a generator object. This object is not a list (it's better) but it behaves like one. You can try iterating through it like this:

for i in zip([1,2,3,4], ['a','b','c','d']):
    print i

The Quick Answer

Please show us the code surrounding where the error happened. But I think I can still give you a quick (and not necessarily good) solution.

turn this

for i in reversed(range(1, len(x))):

into this:

for i in reversed(range(1, len(list(x))):

answered Jun 27 '16 at 04:10

Samie Bencherif

1,285
12
27

Even better for performance is if you can keep using generators, so that you do not build huge lists in memory. This will probably require more thorough refactoring, which is not possible with the code fragments in the question alone. – tripleee Jun 27 '16 at 04:59
FYI: merged from https://stackoverflow.com/questions/38045628/error-while-shuffling-data-for-neural-network – Shog9 Jun 27 '16 at 20:46

score 2 · Answer 5 · edited Nov 09 '18 at 00:06

In mnist_loader wrap your zip results in list() constructs like below

def load_data_wrapper():    
    tr_d, va_d, te_d = load_data()
    training_inputs = [np.reshape(x, (784,1)) for x in tr_d[0]]
    training_results = [vectorized_result(y) for y in tr_d[1]]
    training_data = list(zip(training_inputs, training_results))
    validation_inputs = [np.reshape(x,(784, 1))for x in va_d[0]]
    validation_data = list(zip(validation_inputs, va_d[1]))
    test_inputs = [np.reshape(x, (784, 1)) for x in te_d[0]]
    test_data = list(zip(test_inputs, te_d[1]))
    return(training_data, validation_data, test_data)

This looks like the better choice, as it fixes the problem right where it is originated with the use of zip() — Igor Stoppa, Oct 20 '19 at 14:17

Python 2 --> 3: object of type 'zip' has no len()

5 Answers5

Some Info

The Quick Answer

Linked

Related