0

I've been trying to create the convolutional network from the Tensorflow tutorial, but I've been having trouble. For some reason, I'm getting errors where the size of y_conv is 4x larger than the size of y_, and I have no idea why. I found this question, but it appears to be a different problem than mine, though it looks similar.

To be clear, the batch size in the below code is 50, but the error it's coming up with is

tensorflow.python.framework.errors.InvalidArgumentError: Incompatible shapes: [200] vs. [50]

and when I change the batch size to 10, I get

tensorflow.python.framework.errors.InvalidArgumentError: Incompatible shapes: [40] vs. [10]

so it's related to the batch size somehow, but I can't figure that out. Can anybody tell me what's wrong with this code? It's pretty much straight from the tutorial linked above.

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)

import tensorflow as tf
sess = tf.InteractiveSession()

def weight_variable(shape):
  initial = tf.truncated_normal(shape, stddev=0.1)
  return tf.Variable(initial)

def bias_variable(shape):
  initial = tf.constant(0.1, shape=shape)
  return tf.Variable(initial)

def conv2d(x, W):
  return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')

def max_pool_2x2(x):
  return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides = [1, 2, 2, 1], padding='SAME')

x = tf.placeholder("float", shape=[None, 784])
y_ = tf.placeholder("float", shape=[None, 10])

w_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])
x_image = tf.reshape(x, [-1, 28, 28, 1])
h_conv1 = tf.nn.relu(conv2d(x_image, w_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)

w_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])
h_conv2 = tf.nn.relu(conv2d(h_conv1, w_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

w_fc1 = weight_variable([7*7*64, 1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, w_fc1) + b_fc1)

keep_prob = tf.placeholder("float")
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])

y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)

cross_entropy = -tf.reduce_sum(y_ * tf.log(tf.clip_by_value(y_conv, 1e-10, 1.0)))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
sess.run(tf.initialize_all_variables())
for i in range(20000):
  batch = mnist.train.next_batch(50)
  if i%100 == 0:
    train_accuracy = accuracy.eval(feed_dict={x:batch[0], y_: batch[1], keep_prob: 1.0})
    print("step %d, training accuracy %g"%(i, train_accuracy))
  train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

print("test accuracy %g"%accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))
Cœur
  • 37,241
  • 25
  • 195
  • 267
Beez
  • 391
  • 5
  • 6
  • 16
  • Which operation is generating that error? – mdaoust Jan 16 '16 at 13:14
  • I'm getting one on both correct_prediction and train_accuracy. I assume it's triggering on train_accuracy because that's what's ultimately calling correct_prediction. – Beez Jan 16 '16 at 18:35

1 Answers1

5

The reshapes with the -1's are clues. It's not the batch size that's wrong it's the image size. You're flattening it out into the batch dimension.

Why is the image the wrong size?

On the second conv you're passing conv1 instead of pool1 conv2d(h_conv1, w_conv2).

Personally for pipelines like this I like to use 1 name for the data as it flows through.

Start using a debugger, it's worth it!

mdaoust
  • 6,242
  • 3
  • 28
  • 29
  • I'm still not even sure why they're using -1's, that doesn't make sense to me. But, that's a different problem. Your answer was correct, I had typed it wrong. I'll have to do some research and find a good debugger. – Beez Jan 18 '16 at 01:02