Linear Regression in TensorFlow: How to obtain all estimated y-values of y = m*x?

Question

I am working with the code for linear regression in TensorFlow from https://github.com/nlintz/TensorFlow-Tutorials/blob/master/1_linear_regression.py

This code computes a regression-function that I will call y_estimate. As this is linear regression, the following formula holds: y_estimate = m*x. The coefficient m is equal to the weights of the neural net's layer. Those weights are extracted and we get a perfectly working regression formula.

However, I want to take another approach: First, I want to sample many different values for y_estimate. For example, I want to pass the neural net 101 values for x. Then, I want 101 values for y_estimate from the neural net. Second, I want to plot those values.

Unfortunately, I fail to obtain the values for y_estimate. In the neural net, those values are calcuated through y_model = model(X, w). As X and w both contain many elements (101 to be precise), y_model should also contain 101 elements. I tried different approaches to print all the values of y_model, but each failed.

In the following, I will show my approaches. I only copy the relevant code, the rest is exactly the same as the code in the GIT posted above.

1) First, I just try to print y_model naively:

with tf.Session() as sess:
    tf.initialize_all_variables().run()

    for i in range(100):
        for (x, y) in zip(trX, trY):
            sess.run(train_op, feed_dict={X: x, Y: y})

    print('y_model: ', y_model)
    print(sess.run(w))

Output: ('y_model: ', <tf.Tensor 'Mul_8:0' shape=<unknown> dtype=float32>)

2) Second, I try to fetch the tensor y_model first and then print the result:

with tf.Session() as sess:
    tf.initialize_all_variables().run()

    for i in range(100):
        for (x, y) in zip(trX, trY):
            sess.run(train_op, feed_dict={X: x, Y: y})

    estimates = sess.run(y_model, feed_dict={X: x, Y: y})
    print('estimates: ', estimates)
    print(sess.run(w))

Output: ('estimates: ', 2.0016618)

3) Third, I tried to first use the op tf.Print() that should later print the values of the tensor when evaluating. Note that calling tf.get_default_session().run(t) is equivalent to calling t.eval() as elaborated here.

w = tf.Variable(0.0, name="weights") # create a shared variable (like theano.shared) for the weight matrix
y_model = model(X, w)

cost = tf.square(Y - y_model) # use square error for cost function

train_op = tf.train.GradientDescentOptimizer(0.01).minimize(cost) # construct an optimizer to minimize cost and fit line to my data
temp1 = tf.Print(y_model,[y_model])


with tf.Session() as sess:
    # you need to initialize variables (in this case just variable W)
    tf.initialize_all_variables().run()

    for i in range(100):
        for (x, y) in zip(trX, trY):
            sess.run(train_op, feed_dict={X: x, Y: y})

    estimates = sess.run(temp1, feed_dict={X: x, Y: y})
    print('estimates: ', estimates)
    print(sess.run(w))

Output: ('estimates: ', 2.0458241)

4) A fourth approach would be to use for example tf.reduce_max() to only obtain the maximum value:

with tf.Session() as sess:
    tf.initialize_all_variables().run()

    for i in range(100):
        for (x, y) in zip(trX, trY):
            sess.run(train_op, feed_dict={X: x, Y: y})

    temp1 = tf.reduce_max(y_model)
    estimate_max = sess.run(temp1, feed_dict={X: x, Y: y})
    print('estimate_max: ', estimate_max)
    print(sess.run(w))

Output: ('estimate_max: ', 1.887839)

Now my concrete question: Why do I only get one value instead of 101 values for y_model (regarding my approaches 2 und 3)? Shouldn't y_model return one value for each x of the input?

What did I compute instead and how can I obtain the 101 values i desire?

Thank you very much for your help!

Do you want the values iterative? E.g. the first value is training with 1 input, the second with 2 and so on, until the last one is training on all 101 values? — Jan, May 31 '16 at 12:16
@JeD Short answer: I want all the values at once after all training has completed. This way I can plot the resulting regression function approximately. Long answer: If I am understanding the code correctly, the neural net is trained 100 times (`for i in range(100):`). Each training consists of the calculation of 101 values for y_model and the weights respectively. (`for (x, y) in zip(trX, trY):`). Thus, after each of the 100 trainings there should be 101 y_estimates to display. I want the final ones. — , May 31 '16 at 13:20
Since you are doing a 1-D Linear regression there is only a single value after eacht training, namely the approximated slope. But there are 100 of these as the net is trained 100 times. You don't input all 100 values at once, but one after the other, so there is only a single weight to train. — Jan, May 31 '16 at 14:03
I think the output-value is not the slope (even though it has the same value by chance) but rather f(x) for ONE x. The slope is displayed in the weights, not the output. However, your answer still leads to a working code which I will elaborate below. So thank you! — , May 31 '16 at 18:05

score 0 · Answer 1 · answered May 31 '16 at 18:14

The second approach points in the right direction. However, it is implemented slightly incorrect.

More specifically, the code only prints y_model for one x as the feed dictionary only passes one x. Therefore, one has to just implement a for-loop to ensure that each x is fed to the model. Here is the working code:

with tf.Session() as sess:
    tf.initialize_all_variables().run()

    for i in range(100):
        for (x, y) in zip(trX, trY):
            sess.run(train_op, feed_dict={X: x, Y: y})

    for (x, y) in zip(trX, trY):
        estimates = sess.run(y_model, feed_dict={X: x, Y: y})
        print('estimates: ', estimates)

Linear Regression in TensorFlow: How to obtain all estimated y-values of y = m*x?

1 Answers1