IndexError: list index out of range in TensorFlow

Question

I got an error ,IndexError: list index out of range.Traceback says

Run id: P0W5X0
Log directory: /tmp/tflearn_logs/
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python@2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/xxx/anaconda/xxx/lib/python2.7/site-packages/tflearn/data_flow.py", line 201, in fill_batch_ids_queue
    ids = self.next_batch_ids()
  File "/Users/xxx/anaconda/xxx/lib/python2.7/site-packages/tflearn/data_flow.py", line 215, in next_batch_ids
    batch_start, batch_end = self.batches[self.batch_index]
IndexError: list index out of range

I wrote codes,

# coding: utf-8
import tensorflow as tf
import tflearn

from tflearn.layers.core import input_data,dropout,fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.normalization import local_response_normalization
from tflearn.layers.estimator import regression

tf.reset_default_graph()
net = input_data(shape=[None,20000, 4, 42])
net = conv_2d(net, 4, 16, activation='relu')
net = max_pool_2d(net, 1)
net = tflearn.activations.relu(net)
net = dropout(net, 0.5)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net, optimizer='adam', learning_rate=0.5, loss='categorical_crossentropy')

model = tflearn.DNN(net)

model.fit(np.array(trainDataSet).reshape(1,20000, 4, 42), np.array(trainLabel), n_epoch=400, batch_size=32, validation_set=0.1, show_metric=True)


pred = np.array(model.predict(np.array(testDataSet).reshape(1,20000, 4, 42)).argmax(axis=1))

label = np.array(testLabel).argmax(axis=0)
accuracy = np.mean(pred == label, axis=0)

print(accuracy)

I really cannot understand why such an error happens.I tried to rewrite into

model.fit(np.array(trainDataSet).reshape(1,20000, 4, 42), np.array(trainLabel), n_epoch=400, batch_size=1, validation_set=0.1, show_metric=True)

because bach causes this error,but same error happens.I rewrite another number in this part but same error happens.What is wrong in my codes?How should I fix this?

tell me about the training data set. What is it and why is it shaped the way it is? it appears from input_data(shape=[None,20000, 4, 42]) that you are expecting some number of batches of shape 20000x4x42 but you are feeding it 1 sample of 20000x4x42 in your model.fit. — Anton Codes, Nov 21 '18 at 19:30
I cannot understand what you are saying little bit.`np.array(trainDataSet).shape` is (20000, 4, 42). — user10492592, Nov 22 '18 at 11:47

score 2 · Answer 1 · answered Mar 29 '19 at 17:13

Question

How can I fix my list index out of range error?

Answer

It appears from your code that your training and test set that you are passing into the neural net only has 1 element given by reshape(1,20000, 4, 42) of shape 20000x4x42. What I believe you meant is to have 20000 elements of 4x42.

Instead of reshape(1,20000, 4, 42), let's use reshape(20000, 4, 42, 1). We will also have to change input_data(shape=[None, 20000, 4, 42]) to input_data(shape=[None, 4, 42, 1])

If you do this your code works fine.

Working Code

# coding: utf-8
import tensorflow as tf
import tflearn

from tflearn.layers.core import input_data,dropout,fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.normalization import local_response_normalization
from tflearn.layers.estimator import regression

tf.reset_default_graph()
net = input_data(shape=[None, 4, 42, 1])
net = conv_2d(net, 4, 16, activation='relu')
net = max_pool_2d(net, 1)
net = tflearn.activations.relu(net)
net = dropout(net, 0.5)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net, optimizer='adam', learning_rate=0.5, loss='categorical_crossentropy')

model = tflearn.DNN(net)

model.fit(np.array(trainDataSet).reshape(20000, 4, 42, 1), np.array(trainLabel), n_epoch=400, batch_size=32, validation_set=0.1, show_metric=True)


pred = np.array(model.predict(np.array(testDataSet).reshape(20000, 4, 42, 1)).argmax(axis=1))

label = np.array(testLabel).argmax(axis=0)
accuracy = np.mean(pred == label, axis=0)

print(accuracy)

Output

To get the above code to work we have to include some training and test data. Numpy random is used like so

import numpy as np

trainDataSet = np.random.rand(20000, 4, 42)
trainLabel = ( np.random.rand(20000,2) > .5 ) *1.0

testDataSet = np.random.rand(20000, 4, 42)
testLabel = ( np.random.rand(20000,2) > .5 ) *1.0

Here is the output

Run id: JDSG88
Log directory: /tmp/tflearn_logs/
---------------------------------
Training samples: 18000
Validation samples: 2000
--
Training Step: 563  | total loss: 12.13387 | time: 5.312s
| Adam | epoch: 001 | loss: 12.13387 - acc: 0.7138 | val_loss: 11.90437 - val_acc: 0.7400 -- iter: 18000/18000
--
Training Step: 1126  | total loss: 11.58909 | time: 5.184s
| Adam | epoch: 002 | loss: 11.58909 - acc: 0.7496 | val_loss: 11.90437 - val_acc: 0.7400 -- iter: 18000/18000
--
Training Step: 1689  | total loss: 11.93482 | time: 5.174s
| Adam | epoch: 003 | loss: 11.93482 - acc: 0.7357 | val_loss: 11.90437 - val_acc: 0.7400 -- iter: 18000/18000
--
...

score 0 · Answer 2 · answered Nov 24 '18 at 11:50

I also had the same problem with you. My solution is making the number of n_epoch equal to your row's number of the dataset. For example, my array's shape is 461*5, the value of the n_epoch is 461. you can also make the value a little bit bigger or shorter than your row's number. In my code, 500 or 400 is also useful.

IndexError: list index out of range in TensorFlow

2 Answers2

Question

Answer

Working Code

Output