16

From https://github.com/google/deepdream/blob/master/dream.ipynb

def objective_L2(dst):          # Our training objective. Google has since release a way to load
    dst.diff[:] = dst.data      # arbitrary objectives from other images. We'll go into this later.

def make_step(net, step_size=1.5, end='inception_4c/output', 
              jitter=32, clip=True, objective=objective_L2):
    '''Basic gradient ascent step.'''

    src = net.blobs['data'] # input image is stored in Net's 'data' blob
    dst = net.blobs[end]

    ox, oy = np.random.randint(-jitter, jitter+1, 2)
    src.data[0] = np.roll(np.roll(src.data[0], ox, -1), oy, -2) # apply jitter shift

    net.forward(end=end)
    objective(dst)  # specify the optimization objective
    net.backward(start=end)
    g = src.diff[0]
    # apply normalized ascent step to the input image
    src.data[:] += step_size/np.abs(g).mean() * g

    src.data[0] = np.roll(np.roll(src.data[0], -ox, -1), -oy, -2) # unshift image

    if clip:
        bias = net.transformer.mean['data']
        src.data[:] = np.clip(src.data, -bias, 255-bias)

If I understand what is going on correctly, the input image in net.blobs['data'] is inserted into the NN until the layer end. Once, the forward pass is complete until end, it calculates how "off" the blob at the end is from "something".

Questions

  • What is this "something"? Is it dst.data? I stepped through a debugger and found that dst.data was just a matrix of zeros right after the assignment and then filled with values after the backward pass.

  • Anyways, assuming it finds how "off" the result of the forward pass is, why does it try to do a backwards propagation? I thought the point of deep dream wasn't to further train the model but "morph" the input image into whatever the original model's layer represents.

  • What exactly does src.data[:] += step_size/np.abs(g).mean() * g do? It seems like applying whatever calculation was done above to the original image. Is this line what actually "morphs" the image?

Links that I have already read through

I would be interested in what the author of the accepted answer meant by

we take the original layer blob and "enhance" signals in it. What does it mean, I don't know. Maybe they just multiply the values by coefficient, maybe something else.

In this blog post the author comments next to src.data[:] += step_size/np.abs(g).mean() * g: "get closer to our target data." I'm not too clear what "target data" means here.

Community
  • 1
  • 1
Kent Shikama
  • 3,910
  • 3
  • 22
  • 55

0 Answers0