4

I am learning tensorflow using the tensorflow machine learning cookbook (https://github.com/nfmcclure/tensorflow_cookbook). I am currently on the NLP chapter (07). I am very confused about how one decides on the dimensions of the tensorflow variables. For example, in the bag of words example they use:

# Create variables for logistic regression
A = tf.Variable(tf.random_normal(shape=[embedding_size,1]))
b = tf.Variable(tf.random_normal(shape=[1,1]))

# Initialize placeholders
x_data = tf.placeholder(shape=[sentence_size], dtype=tf.int32)
y_target = tf.placeholder(shape=[1, 1], dtype=tf.float32)

and in the tf-idf example they use:

# Create variables for logistic regression
A = tf.Variable(tf.random_normal(shape=[max_features,1]))
b = tf.Variable(tf.random_normal(shape=[1,1]))

x_data = tf.placeholder(shape=[None, max_features], dtype=tf.float32)
y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)

How does one decide on when to use None vs. 1 in the placeholder shapes? Thank you!

user3490622
  • 939
  • 2
  • 11
  • 30
  • Take a look at [this answer](https://stackoverflow.com/questions/37096225/how-to-understand-static-shape-and-dynamic-shape-in-tensorflow) for an explanation of how shapes are handled in Tensorflow – GPhilo Aug 14 '17 at 14:07

2 Answers2

4

Using None as part of a shape, means it will be determined when you run the session. This is useful for training in what is called batch training where you feed each iteration of the training process a fixed size subset of the data. So if you kept it at None you can switch between batch sizes without a problem. (Although you won't be doing so in the same session, but every session you can try a different batch size)

When you state a specific shape, that is what it will be and that is the only shape that can be fed to it during the session (using the feed_dict param)

In your specific example, the first part of code, the shape of y_target will always be [1, 1] where in the second part of code, y_target could be [10, 1] / [200, 1] / [<whatever>, 1]

bluesummers
  • 11,365
  • 8
  • 72
  • 108
1

'None' should be used when count of elements in placeholder is unknown in advance. But for example in x_data placeholder if count of data elements is 1 i.e. it is known in advance, then you can replace 'None' with 1.

Gaurav Pawar
  • 449
  • 2
  • 7