Quite trivially:
def build_graph():
cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(...)
train_op = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
return cross_entropy, train_op # both tensorflow OPs
cross_entropy, train_op = build_graph()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
# Evaluate and return cross_entropy, and execute the train_op (update the weights)
result = sess.run([cross_entropy, train_op], feed_dict={...})
print(result[0]) # The value of the cross entropy loss function
There are many excellent tutorials here: https://github.com/aymericdamien/TensorFlow-Examples
You'll find them doing exactly this in full working models.
If you don't have access to the tensor, then you can look it up in the graph by name as such:
tf.get_default_graph().get_tensor_by_name("example:0")
See this question: Tensorflow: How to get a tensor by name?
Note that if you didn't name your tensors well, this is going to be a royal pain in the rear, so, it's one of many good reasons to name your tensors well. The default name of a tensor will use some reference to the operation, colon, an index number, such as "add:2" for the 3rd add operation.
You can get a list of all tensors in a graph with:
[n.name for n in tf.get_default_graph().as_graph_def().node]
That code is copied from from this question: In Tensorflow, get the names of all the Tensors in a graph
Responding to this follow up question in comments:
I would like to know which one is the train_op optimizing without
having to name them with a specific name. So given a train_op object,
is there any way of retrieving the tensor (or the name of the tensor)
which represents the last value that train_op is minimizing? I need it
because I am automatizing a set of unit tests so that if I plug a
tensorflow model to my system it automatically finds, given the
optimizers, the tensors that represent the loss (that way I can
automatically perform gradient checks).
I have coded a gradient descent optimizer as part of my research. Here are a few ideas you might consider:
1) Here's a link to the optimizer I followed when I did the same: https://github.com/openai/iaf/blob/master/tf_utils/adamax.py That's implementing AdaMax in python. You'll be interested in _apply_dense() which takes a gradient and its variable and performs the update. It's called for each trainable variable. Note that most optimizers in tensorflow are coded in C, not using the python interface. So I'm not sure if this will help or not, but understanding the process better can't be a bad thing.
2) You can get the gradient of any variable with respect to any other variable. So you could grab the collection of trainable variables with tf.trainable_variables() and then call tf.gradients to get the gradients of the trainable variables with respect to the loss function. You would need the loss function for this rather than the train OP though. I expect you can find the loss automatically from the optimizer.
If you're just trying to find the loss function from the train OP you might find what you need by following the graph dependencies as is described in this question: How can I list all Tensorflow variables a node depends on?
Here's a way I've used before to get a list of each variable and its inputs and outputs. I suspect you could figure out how to traverse this datastructure to find what you need.
tf.get_default_graph().as_graph_def()
Out[6]:
node {
name: "x"
op: "Const"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_FLOAT
tensor_shape {
}
float_val: 42.0
}
}
}
}
