1

I am new to TensorFlow and neural networks. I am trying to build a neural network that can classify images in the CIFAR-10 dataset.

Here is my code:

import tensorflow as tf
import pickle
import numpy as np
import random

image_size= 32*32*3 # because 3 channels
n_classes = 10
lay1_size = 50
batch_size = 100

def unpickle(filename):
    with open(filename,'rb') as f:
        data = pickle.load(f, encoding='latin1')
    x = data['data']
    y = data['labels']
    # shuffle the data
    z = list(zip(x,y))
    random.shuffle(z)
    x, y = zip(*z)
    x = x[:batch_size]
    y = y[:batch_size]
    # covert decimals to one hot arrays
    y = np.eye(n_classes)[[y]]
    return x, y

# set up network
def add_layer(inputs, in_size, out_size, activation_function=None):
    W = tf.Variable(tf.random_normal([in_size, out_size]), dtype=tf.float32)
    b = tf.Variable(tf.zeros([1,out_size]) + 0.1, dtype=tf.float32)
    Wx_plus_b = tf.matmul(inputs, W) + b
    if activation_function is None:
        output = Wx_plus_b
    else:
        output = activation_function(Wx_plus_b)
    return output

def compute_accuracy(v_xs, v_ys):
    global prediction
    y_pre = sess.run(prediction, feed_dict={xs:v_xs})
    correct_prediction = tf.equal(tf.argmax(y_pre,1), tf.argmax(v_ys, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
    result = sess.run(accuracy, feed_dict={xs:v_xs, ys:v_ys})
    return result

xs = tf.placeholder(tf.float32, [None,image_size])
ys = tf.placeholder(tf.float32)

lay1 = add_layer(xs, image_size, lay1_size, activation_function=tf.nn.tanh)

lay2 = add_layer(lay1, lay1_size, lay1_size, activation_function=tf.nn.tanh)

prediction = add_layer(lay2, lay1_size, n_classes, activation_function=tf.nn.softmax)

cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys*tf.log(prediction), reduction_indices=[1]))

train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
#train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
# run network
sess = tf.Session()
sess.run(tf.initialize_all_variables())

x_test, y_test = unpickle('test_batch')
for i in range(1000):
    x_train, y_train = unpickle('data_batch_1')
    sess.run(train_step, feed_dict={xs:x_train,ys:y_train})
    if i % 50 == 0:
        print(compute_accuracy(x_test, y_test)) 
sess.close()

I am using two hidden layers with 50 nodes in each layer. I am running 1,000 cycles, where in each cycle I shuffle data in the dataset and pick the first 100 images of that shuffle to train on.

I am consistently getting ~0.1 accuracy, the machine is not learning at all.

When I modify the code to use the MNIST dataset instead of the CIFAR-10 dataset I get ~0.87 accuracy.

I took code from an MNIST tutorial and am trying to modify it to classify CIFAR-10 data.

I can't figure out what's wrong here. How do I get my algorithm to learn?

dda
  • 6,030
  • 2
  • 25
  • 34
Matt
  • 2,232
  • 8
  • 35
  • 64
  • Why are you computing cross entropy manually? Why are you shuffling the data set instead of going through it sequentially? – Natecat Jan 02 '17 at 08:05
  • @Natecat Is there a function to do that? – Matt Jan 02 '17 at 08:07
  • `tf.nn.softmax_cross_entropy_with_logits` [Here's why to use that instead](http://stackoverflow.com/a/34243720/1357527) – Natecat Jan 02 '17 at 08:09
  • @Natecat I'm still getting ~0.1 accuracy, do you have any suggestions? – Matt Jan 02 '17 at 08:12
  • Are you going through the data sequentially? Also, what are the sizes of your train and test datasets – Natecat Jan 02 '17 at 08:14
  • @Natecat both my train and test datasets are dicts. they contain data that is a 10000*3072 numpy array and 10000 labels with numbers 0-9. I don't know what you mean by going through the data sequentially – Matt Jan 02 '17 at 08:22
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/132073/discussion-between-natecat-and-matt-d). – Natecat Jan 02 '17 at 08:53
  • Shouldn't the `ys = tf.placeholder(tf.float32)` be initialized with a 10 dimensional shape like this: `ys = tf.placeholder(tf.float32, shape=[None, 10])` – Christian Safka Jan 02 '17 at 13:50
  • @Natecat why is it better to loop trough the data sequentially? – Hello Lili Sep 08 '17 at 08:48

0 Answers0