1

I'm just beginning to learn TensorFlow and I have some problems with it. I read the paper -- DEEP COMPRESSION: COMPRESSING DEEP NEURAL NETWORKS WITH PRUNING, TRAINED QUANTIZATION AND HUFFMAN CODING a few days ago. In pruning part, the authors start by learning the connectivity via normal network training. Next, they prune the small-weight connections: all connections with weights below a threshold are removed from the network. Finally,they retrain the network to learn the final weights for the remaining sparse connections.

I want to get all the weights of every layer and compare to the threshold one by one and set the small weights zero.Here is my code and there is a exception TypeError("Using a tf.Tensor as a Python bool is not allowed."). How can I get the value of the weight tensor when I design the network? Has anyone implemented this code yet or any other method suggested? Thanks in advance!

with tf.variable_scope('layer1-conv1'):
    conv1_weights = tf.get_variable(
        "weight", [CONV1_SIZE, CONV1_SIZE, NUM_CHANNELS, CONV1_DEEP],
        initializer = tf.truncated_normal_initializer(stddev=0.1)
    )

    shapeDim=CONV1_SIZE*CONV1_SIZE*NUM_CHANNELS*CONV1_DEEP
    reshape_w=tf.reshape(conv1_weights,[-1])

    i=0
    if step != 0 and step != 1:
        while i < shapeDim:
            if reshape_w[i] < RATIO:
                reshape_w[i] = 0

    conv1_weights=tf.reshape(reshape_w, [CONV1_SIZE, CONV1_SIZE, NUM_CHANNELS, CONV1_DEEP])

    conv1_biases = tf.get_variable("bias", [CONV1_DEEP], initializer=tf.constant_initializer(0.0))

    conv1 = tf.nn.conv2d(input_tensor, conv1_weights, strides=[1, 1, 1, 1], padding='SAME')

    relu1 = tf.nn.relu(tf.nn.bias_add(conv1, conv1_biases))

R.Nancy
  • 61
  • 3

1 Answers1

0

Instead of going through element-by-element, you can use tf.less to compare the whole matrix at once, and then assign based on the boolean matrix.

The following answer should give you a better idea of how to do this:

Conditional assignment of tensor values in TensorFlow

pwp2
  • 451
  • 3
  • 12
  • I’m very grateful for your reply.It's very helpful. But the last statement does not appear to be running. I don't know how to fix it.Here is my code: `compare_matrix = tf.constant(RATIO,shape=[CONV1_SIZE, CONV1_SIZE, NUM_CHANNELS, CONV1_DEEP]) compare_bool=tf.less(conv1_weights,compare_matrix) conv1_weights.assign(tf.where(compare_bool,tf.zeros_like(conv1_weights),conv1_weights))` – R.Nancy Jul 13 '17 at 03:06
  • The 'session.run' statement is defined in another file, is there a problem? – R.Nancy Jul 13 '17 at 03:09