In Tensorflow, get the names of all the Tensors in a graph

Question

I am creating neural nets with Tensorflow and skflow; for some reason I want to get the values of some inner tensors for a given input, so I am using myClassifier.get_layer_value(input, "tensorName"), myClassifier being a skflow.estimators.TensorFlowEstimator.

However, I find it difficult to find the correct syntax of the tensor name, even knowing its name (and I'm getting confused between operation and tensors), so I'm using tensorboard to plot the graph and look for the name.

Is there a way to enumerate all the tensors in a graph without using tensorboard?

score 203 · Accepted Answer · answered Apr 27 '16 at 15:10

203

You can do

[n.name for n in tf.get_default_graph().as_graph_def().node]

Also, if you are prototyping in an IPython notebook, you can show the graph directly in notebook, see show_graph function in Alexander's Deep Dream notebook

answered Apr 27 '16 at 15:10

Yaroslav Bulatov

57,332
22
139
197

2

You can filter this for e.g. variables by adding `if "Variable" in n.op` at the end of the comprehension. – Radu Jul 10 '17 at 10:54
Is there a way to get a specific node if you know the name? – Rocket Pingu Apr 13 '18 at 11:02
To read more about graph nodes: https://www.tensorflow.org/extend/tool_developers/#nodes – Ivan Talalaev Oct 13 '18 at 08:20
4

The command above yields the names of all operations/nodes. To get the names of all tensors, do: tensors_per_node = [node.values() for node in graph.get_operations()] tensor_names = [tensor.name for tensors in tensors_per_node for tensor in tensors] – gebbissimo Jan 21 '19 at 14:53

Szabolcs · Answer 2 · 2021-02-09T12:45:55.990

I'll try to summarize the answers:

To get all nodes in the graph: (type tensorflow.core.framework.node_def_pb2.NodeDef)

all_nodes = [n for n in tf.get_default_graph().as_graph_def().node]

To get all ops in the graph: (type tensorflow.python.framework.ops.Operation)

all_ops = tf.get_default_graph().get_operations()

To get all variables in the graph: (type tensorflow.python.ops.resource_variable_ops.ResourceVariable)

all_vars = tf.global_variables()

To get all tensors in the graph: (type tensorflow.python.framework.ops.Tensor)

all_tensors = [tensor for op in tf.get_default_graph().get_operations() for tensor in op.values()]

To get all placeholders in the graph: (type tensorflow.python.framework.ops.Tensor)

all_placeholders = [placeholder for op in tf.get_default_graph().get_operations() if op.type=='Placeholder' for placeholder in op.values()]

Tensorflow 2

To get the graph in Tensorflow 2, instead of tf.get_default_graph() you need to instantiate a tf.function first and access the graph attribute, for example:

graph = func.get_concrete_function().graph

where func is a tf.function

Heads up for that TF2 version! – ibarrond Mar 08 '21 at 12:11 — ibarrond, Mar 08 '21 at 12:11

score 25 · Answer 3 · edited Jan 23 '19 at 13:33

25

There is a way to do it slightly faster than in Yaroslav's answer by using get_operations. Here is a quick example:

import tensorflow as tf

a = tf.constant(1.3, name='const_a')
b = tf.Variable(3.1, name='variable_b')
c = tf.add(a, b, name='addition')
d = tf.multiply(c, a, name='multiply')

for op in tf.get_default_graph().get_operations():
    print(str(op.name))

edited Jan 23 '19 at 13:33

Mayou36

4,613
2
17
20

answered May 03 '17 at 06:22

Salvador Dali

214,103
147
703
753

3

You cannot get Tensors using `tf.get_operations()`. Only operation you can get. – Soulduck Mar 20 '19 at 07:43
@Soulduck you can get the Tensors for each operation using `op.values()`, for example: `last_tensor = graph.get_operations()[-1].values()` where `graph.get_operations()[-1]` is the last operation in the graph. – Youcef4k Apr 10 '22 at 13:24

score 11 · Answer 4 · edited Mar 28 '17 at 06:53

11

tf.all_variables() can get you the information you want.

Also, this commit made today in TensorFlow Learn that provides a function get_variable_names in estimator that you can use to retrieve all variable names easily.

edited Mar 28 '17 at 06:53

Aaron

2,354
1
17
25

answered Apr 28 '16 at 21:47

Yuan Tang

696
4
15

2

This function is deprecated – CAFEBABE Mar 27 '17 at 22:20
8

... and its successor is `tf.global_variables()` – bluenote10 Apr 18 '17 at 12:29
11

this fetches only variables, not tensors. – Rajarshee Mitra Jul 04 '17 at 10:51
In Tensorflow 1.9.0 shows that `all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02` – stackoverYC Aug 22 '18 at 08:42
`module 'tensorflow' has no attribute 'all_variables'` – skytree Jan 25 '22 at 05:34

score 5 · Answer 5 · answered Mar 02 '18 at 00:53

5

I think this will do too:

print(tf.contrib.graph_editor.get_tensors(tf.get_default_graph()))

But compared with Salvado and Yaroslav's answers, I don't know which one is better.

answered Mar 02 '18 at 00:53

Lu Howyou

51
1
2

This one worked with a graph imported from a frozen_inference_graph.pb file used in the tensorflow object detection API. Thanks – simo23 Aug 27 '18 at 13:51

score 5 · Answer 6 · edited Dec 20 '18 at 14:22

The accepted answer only gives you a list of strings with the names. I prefer a different approach, which gives you (almost) direct access to the tensors:

graph = tf.get_default_graph()
list_of_tuples = [op.values() for op in graph.get_operations()]

list_of_tuples now contains every tensor, each within a tuple. You could also adapt it to get the tensors directly:

graph = tf.get_default_graph()
list_of_tuples = [op.values()[0] for op in graph.get_operations()]

gebbissimo · Answer 7 · 2019-01-21T14:52:13.980

4

Since the OP asked for the list of the tensors instead of the list of operations/nodes, the code should be slightly different:

graph = tf.get_default_graph()    
tensors_per_node = [node.values() for node in graph.get_operations()]
tensor_names = [tensor.name for tensors in tensors_per_node for tensor in tensors]

edited Jan 21 '19 at 14:52

answered Jan 21 '19 at 14:38

gebbissimo

2,137
2
25
35

score 3 · Answer 8 · answered May 15 '18 at 09:06

Previous answers are good, I'd just like to share a utility function I wrote to select Tensors from a graph:

def get_graph_op(graph, and_conds=None, op='and', or_conds=None):
    """Selects nodes' names in the graph if:
    - The name contains all items in and_conds
    - OR/AND depending on op
    - The name contains any item in or_conds

    Condition starting with a "!" are negated.
    Returns all ops if no optional arguments is given.

    Args:
        graph (tf.Graph): The graph containing sought tensors
        and_conds (list(str)), optional): Defaults to None.
            "and" conditions
        op (str, optional): Defaults to 'and'. 
            How to link the and_conds and or_conds:
            with an 'and' or an 'or'
        or_conds (list(str), optional): Defaults to None.
            "or conditions"

    Returns:
        list(str): list of relevant tensor names
    """
    assert op in {'and', 'or'}

    if and_conds is None:
        and_conds = ['']
    if or_conds is None:
        or_conds = ['']

    node_names = [n.name for n in graph.as_graph_def().node]

    ands = {
        n for n in node_names
        if all(
            cond in n if '!' not in cond
            else cond[1:] not in n
            for cond in and_conds
        )}

    ors = {
        n for n in node_names
        if any(
            cond in n if '!' not in cond
            else cond[1:] not in n
            for cond in or_conds
        )}

    if op == 'and':
        return [
            n for n in node_names
            if n in ands.intersection(ors)
        ]
    elif op == 'or':
        return [
            n for n in node_names
            if n in ands.union(ors)
        ]

So if you have a graph with ops:

['model/classifier/dense/kernel',
'model/classifier/dense/kernel/Assign',
'model/classifier/dense/kernel/read',
'model/classifier/dense/bias',
'model/classifier/dense/bias/Assign',
'model/classifier/dense/bias/read',
'model/classifier/dense/MatMul',
'model/classifier/dense/BiasAdd',
'model/classifier/ArgMax/dimension',
'model/classifier/ArgMax']

Then running

get_graph_op(tf.get_default_graph(), ['dense', '!kernel'], 'or', ['Assign'])

returns:

['model/classifier/dense/kernel/Assign',
'model/classifier/dense/bias',
'model/classifier/dense/bias/Assign',
'model/classifier/dense/bias/read',
'model/classifier/dense/MatMul',
'model/classifier/dense/BiasAdd']

score 1 · Answer 9 · answered Sep 23 '20 at 14:25

The following solution works for me in TensorFlow 2.3 -

def load_pb(path_to_pb):
    with tf.io.gfile.GFile(path_to_pb, 'rb') as f:
        graph_def = tf.compat.v1.GraphDef()
        graph_def.ParseFromString(f.read())
    with tf.Graph().as_default() as graph:
        tf.import_graph_def(graph_def, name='')
        return graph

tf_graph = load_pb(MODEL_FILE)
sess = tf.compat.v1.Session(graph=tf_graph)

# Show tensor names in graph
for op in tf_graph.get_operations():
    print(op.values())

where MODEL_FILE is the path to your frozen graph.

Taken from here.

score 0 · Answer 10 · answered Dec 18 '18 at 05:46

0

This worked for me:

for n in tf.get_default_graph().as_graph_def().node:
    print('\n',n)

answered Dec 18 '18 at 05:46

Akshaya Natarajan

1,865
15
17

In Tensorflow, get the names of all the Tensors in a graph

10 Answers10

Linked