0

I am aware that I can visualize the weights of the layers in a histogram using tensorboard Understanding TensorBoard (weight) histograms

My question, is it possible to "split" a fully connected layer into two separate histograms ? Because I have input coming from 2 sources that is concatenated before before going through a fully connected layer and I want to see the weight distribution for the 2 sources. Below I have a simple example where a and b are concatenated before being passed through a fully connected layer.

a is of size 1024 and b of size 256. The out layer has 1024 units.

out = tf.matmul(tf.concat(values=(a, b), axis=1), weight) + bias
Kong
  • 2,202
  • 8
  • 28
  • 56

1 Answers1

1

Assuming your weight to have shape 1280 x 1024, you can first split your weight as

weight_a = tf.slice(weight, [0, 0], [1024, 1024])
weight_b = tf.slice(weight, [1024, 0], [1280, 1024])

Now, you can visualize weight_a and weight_b.

The slicing can be generalized as well but since you explicitly specified the size of each tensor, the above is the quickest method.

layog
  • 4,661
  • 1
  • 28
  • 30