keras: concatenate two images as input (DeepVO)

Question

I'm trying to implement the model described in this paper.
One item I've having trouble with is setting up the input, which is supposed to be two images stacked, meaning, I have a set of consecutive (i & i+1) images 2048x2048x1 (monochrome), so the input tensor would be 2048x2048x2, but each successive input to the neural net would be the following set of images (i+1 & i+2). So far I have

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Concatenate, Activation, Conv2D, MaxPooling2D, Flatten, Input
from keras.klayers import Embedding,LSTM 

inp1 = Input((2048,2048,1))
inp2 = Input((2048,2048,1))
deepVO = Sequential()
deepVO.add(Concatenate(inp1,inp2,-1))
deepVO.add(Conv2D(64,(2,2)))
deepVO.add(Activation('relu'))
#....continue to add other layers

The error I get at deepVO_CNN.add(Concatenate(inp1,inp2,-1)) is:

TypeError: __init__() takes from 1 to 2 positional arguments but 4 were given.

Possible duplicate of [How to concatenate two layers in keras?](https://stackoverflow.com/questions/43196636/how-to-concatenate-two-layers-in-keras) — Souradeep Nanda, Mar 02 '18 at 03:36
Why? Just use `Input((2048,2048,2))` and pass both the images concatenated using numpy. — Souradeep Nanda, Mar 02 '18 at 03:40
You are mixing sequential and functional APIs, don't do that. For this you can just use the functional API. — Dr. Snoopy, Mar 02 '18 at 08:17

score 6 · Answer 1 · answered Mar 02 '18 at 08:22

try the keras api mode like this:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Activation, Conv2D, MaxPooling2D, Flatten, Input, concatenate

from keras.models import Model

inp1 = Input((2048,2048,1))
inp2 = Input((2048,2048,1))

deepVO = concatenate([inp1, inp2],axis=-1)
deepVO = Conv2D(64,(2,2))(deepVO)
deepVO = Activation('relu')(deepVO)
...

...
outputs = Dense(num_classes, activation='softmax')(deepVO)
deepVO = Model([inp1, inp2], outputs)
#deepVO.summary()

keras: concatenate two images as input (DeepVO)

1 Answers1