6

I've built a neural network that performs reasonably well, and I'd like to replicate my model in a non-Python environment. I set up my network as follows:

sess = tf.InteractiveSession()
x = tf.placeholder(tf.float32, shape=[None, 23])
y_ = tf.placeholder(tf.float32, shape=[None, 2])
W = tf.Variable(tf.zeros([23,2]))
b = tf.Variable(tf.zeros([2]))
sess.run(tf.initialize_all_variables())
y = tf.nn.softmax(tf.matmul(x,W) + b)

How can I obtain a decipherable .csv or .txt of my weights and biases?

EDIT: Below is my full script:

import csv
import numpy
import tensorflow as tf

data = list(csv.reader(open("/Users/sjayaram/developer/TestApp/out/production/TestApp/data.csv")))
[[float(j) for j in i] for i in data]
numpy.random.shuffle(data)
results=data

#delete results from data
data = numpy.delete(data, [23, 24], 1)
#delete data from results
results = numpy.delete(results, range(23), 1)

sess = tf.InteractiveSession()
x = tf.placeholder(tf.float32, shape=[None, 23])
y_ = tf.placeholder(tf.float32, shape=[None, 2])
W = tf.Variable(tf.zeros([23,2]))
b = tf.Variable(tf.zeros([2]))
sess.run(tf.initialize_all_variables())
y = tf.nn.softmax(tf.matmul(x,W) + b)
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

#train the model, saving 80 entries for testing
#batch-size: 40
for i in range(0, 3680, 40):
  train_step.run(feed_dict={x: data[i:i+40], y_: results[i:i+40]})

correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(accuracy.eval(feed_dict={x: data[3680:], y_: results[3680:]}))
surf-n-cherf
  • 63
  • 1
  • 7

1 Answers1

5

You can fetch the variables as NumPy arrays, and use numpy.savetxt() to write out the contents as text or CSV:

import numpy as np

W_val, b_val = sess.run([W, b])

np.savetxt("W.csv", W_val, delimiter=",")
np.savetxt("b.csv", b_val, delimiter=",")

Note that this is unlikely to give performance as good as using TensorFlow's native replication mechanisms, in the distributed runtime.

mrry
  • 125,488
  • 26
  • 399
  • 400
  • When I run these lines, the .csv files produced don't contain any numerical data, just "nan" (not a number) – surf-n-cherf Aug 02 '16 at 02:44
  • What do you get if you simply `print` the numpy arrays? (The most likely explanation is that your weights have gone to NaN, due to exploding gradients etc., but that shouldn't happen with your small sample program.) – mrry Aug 02 '16 at 02:49
  • I get the same thing, arrays of "nan." – surf-n-cherf Aug 02 '16 at 02:57
  • Do you get the same after if you run the program in your question, followed by the code in my answer? – mrry Aug 02 '16 at 05:37
  • Yes, I get the same output... I've edited my question to include my entire script. Maybe there's an error there? – surf-n-cherf Aug 02 '16 at 17:44
  • That formulation for `cross_entropy` is unfortunately numerically unstable, and often leads to NaNsl. [This answer](http://stackoverflow.com/a/33713196/3574081) has a solution that should avoid divergence. Alternatively, a [more efficient version](http://stackoverflow.com/a/34243720/3574081) involves using `tf.nn.softmax_cross_entropy_with_logits()` to replace the softmax *and* cross-entropy calculations. – mrry Aug 02 '16 at 17:49
  • Thanks! I made the changes suggested by the first link, and now everything's working. I'll give logits a shot once I'm a bit more familiar with NN and tensorflow. – surf-n-cherf Aug 02 '16 at 19:36