Tensorflow dataset with multiple inputs and target

Question

I am trying to implement a model with the ArcFace Layer: https://github.com/4uiiurz1/keras-arcface

to this extend I created a tf.data.dataset like so:

images= tf.data.Dataset.from_tensor_slices(train.A_image.to_numpy())
target = tf.keras.utils.to_categorical(
    train.Label.to_numpy(), num_classes=n_class, dtype='float32'
)
target = tf.data.Dataset.from_tensor_slices(target)

images= images.map(transform_img)

dataset = tf.data.Dataset.zip((images, target, target))

when I call model.fit(dataset)

I get the following error:

ValueError: Layer model expects 2 input(s), but it received 1 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=<unknown> dtype=float32>]

But this should work according:

tf.data with multiple inputs / outputs in Keras

Can someone point out my folly?

Thanks!

Edit: this solves some problems:

#reads in filepaths to images from dataframe train
images = tf.data.Dataset.from_tensor_slices(train.image.to_numpy())
#converts labels to one hot encoding vector
target = tf.keras.utils.to_categorical(train.Label.to_numpy(), num_classes=n_class, dtype='float32')
#reads in the image and resizes it
images= images.map(transform_img)
input_1 = tf.data.Dataset.zip((anchors, target))
dataset = tf.data.Dataset.zip((input_1, target))

And I think it's what we are trying. But I get a shape error for targets, it's (n_class, 1) instead of just (n_class,)

I.e. the fit methods throws this error

ValueError: Shapes (n_class, 1) and (n_class, n_class) are incompatible

and this warning

input expected is (None, n_class) but received an input of (n_class, 1)

let say you have test data with no labels, how would you do it then. I have the same problem. for test data. my model takes 2 inputs and getting the same error as you.(expected 2 inputs got 1) — tikendraw, Nov 18 '22 at 01:13

Edwin Cheong · Accepted Answer · 2021-05-21T16:12:03.530

8

I've made changes to the solution based on the arcface, you've wanted here is the code, i've managed to train it

The first one is from tensor slices as the original input and i used mnist to test it out

def map_data(inputs, outputs):
    image = tf.cast(inputs['image_input'], tf.float32)
    image = image / 255.
    image = tf.expand_dims(image, axis=2)
    
    labels = tf.one_hot(outputs, 10)
    
    return {'image_input': image, 'label_input': labels}, labels

dataset = tf.data.Dataset.from_tensor_slices(({
    'image_input': x_train, 'label_input': y_train
}, y_train))
dataset = dataset.map(map_data)
dataset = dataset.batch(2)

Here is the second type i have tried using a normal from tensor slices then i converted it to a multiple input, since both the normal labels are used for both the input and output

def map_data(images, annot_labels):
    image = tf.cast(images, tf.float32)
    image = image / 255.
    image = tf.expand_dims(image, axis=2) # convert to 0 - 1 range
    
    labels = tf.one_hot(annot_labels, 10)
    
    return {'image_input': image, 'label_input': labels}, labels

dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
dataset = dataset.map(map_data)
dataset = dataset.batch(2)

edited May 21 '21 at 16:12

answered May 21 '21 at 15:39

Edwin Cheong

879
2
7
12

Hi Edwin, nice to see you again, tried this too (actually in part as a consequence of our discussion yesterday but for a different model) But when I do this, it says it expected two inputs but only received 1 tensor – Olli May 21 '21 at 15:43
Hmm i see, hold on let me try something – Edwin Cheong May 21 '21 at 15:46
This method actually throws the same shape error as my edit above: `....but it was called on an input with incompatible shape (1162, 1).` edit: same warning but different error in the end – Olli May 21 '21 at 15:46
1

Hi Olli, I have runned this with the arcface layer you tried to implement, this 1 is working according to my test – Edwin Cheong May 21 '21 at 16:12
Oh, thank you very much, I'll trust your word for it then and shamelessly copy-paste now. Thank you so much, I was wondering already if I may not have just reverted to generators. But I am so glad I'm learning tf.data.datasets now, it's considerably faster – Olli May 21 '21 at 16:23
1

Btw i am using mnist for this i am not sure it fits your dataset, so do make changes accordingly – Edwin Cheong May 21 '21 at 16:25
Thanks, yes I'll adjust – Olli May 21 '21 at 16:26
1

this worked beautifully I can't thank you enough, I've spent the better part of today working through this. Thank you!! If you're ever in zurich I'll buy you a beer – Olli May 21 '21 at 16:50

score 3 · Answer 2 · answered May 21 '21 at 15:02

3

I think you should do it like this:

target = tf.keras.utils.to_categorical(train.Label.to_numpy(), num_classes=n_class, dtype='float32')
    
images_target = tf.data.Dataset.from_tensor_slices((train.A_image.to_numpy(), target))

images_target = images_target.map(lambda x, y: (transform_img(x), y))
    

target = tf.data.Dataset.from_tensor_slices(target)
    
dataset = tf.data.Dataset.zip((images_target, target))

answered May 21 '21 at 15:02

fgarciador

68
7

Thank you very much. I found that solution too, zipping the former and then the latter, at least almost the same. But then I get en error in the shape of target from the first zip, it adds an extra dimension in the end – Olli May 21 '21 at 15:06
I also just tried your way with lambda (which is slightly different) get the following error: TypeError: () missing 1 required positional argument: 'y' Is it possibly the result of target being one hot encoded? I.e. more than 1 input? – Olli May 21 '21 at 15:10
1

maybe `lambda x: (transform_img(x[0]), x[1])` works. Oh maybe it is the OE thing, not sure about that. – fgarciador May 21 '21 at 15:13
1

Could you edit your question with what you are trying? It's hard to read through comments. – fgarciador May 21 '21 at 15:31
Of course, thank you for your patience. I am still trying to solve the same thing, just made some progress, in line with what you suggested I think. I edited my question above to include that. – Olli May 21 '21 at 15:37
1

Are you adding the batch size after that? something like `dataset = dataset.batch(32)`, just to rule out some options. – fgarciador May 21 '21 at 15:42
Thanks, no, I did originally but now I put away everything and just do model.fit(dataset) – Olli May 21 '21 at 15:44

Tensorflow dataset with multiple inputs and target

2 Answers2