5

I am trying to create a multi-view convolutional neural network that starts off separately applying convolutional and pooling layers to each of the inputs. The structure would look something like this.

CNN Multi View Structure

Just as an example, I would want to have the top, bottom, left, and right view of a cat, apply convolutional and pooling layers to each separately, and then combine this info later.

I have been mostly using Keras on this project, however I cannot seem to figure this out with what Keras has to offer. Is there any way to do this with Keras, and if not, how can I create the first step of this multi-view CNN?

eforkin
  • 107
  • 1
  • 1
  • 9

1 Answers1

9

Take a look at the Keras multi-input multi-output model example. Just follow it and make minor changes. You should be able to adapt it to your usage.

pyan
  • 3,577
  • 4
  • 23
  • 36
  • 2
    I've taken a look at that example before, but it would be nice to have a specific multi-image CNN model example so I can understand how to preproccess the dataset as well. It's difficult from the documentation to determine how to use ImageDataGenerator with a multi image CNN – eforkin Jun 06 '17 at 16:54
  • It will be easier to start with multiple generators. Once you are more experienced, try to write your own customized generator. For using multiple generators, please refer to the example of transforming images and masks together at the bottom of https://keras.io/preprocessing/image/ – pyan Jun 06 '17 at 20:29