I want to build a generator function to input Image Data together with the corresponding metadata. There are more than 20 k Images and that is why I can't just read them into memory. I have tried also to use flow_from_dataframe but in the multi-input case it's getting complicated.
The generator function looks like this:
data_files_generator_train <- function(dir) {
files <- list.files(dir ,pattern = "jpg$")
next_file <- 0
function() {
next_file <<- next_file + 1
if (next_file > length(files))
{next_file <<- 1}
# determine the file name
file <- files[[next_file]]
image <- image_load(file, target_size = c(150,150)) %>%
image_to_array() %>%
array_reshape(dim = c(150, 150, 3)) %>%
imagenet_preprocess_input()
x_image <- image/ 255
x_attr <- paintings_attributes_train %>% filter(path == file) %>% select(-c(target, path))
y <- paintings_attributes_train %>% filter(path == file) %>% select(target)
x_attr <- matrix(x_attr, ncol = 16)
y <- matrix(y)
return(list(list(x_image, x_attr), y))
}}
I get the following error:
Error in py_call_impl(callable, dots$args, dots$keywords) : ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [array([[0.00769116, 0.17699115, 0.1436
the model definition looks as follows:
vision_model <- keras_model_sequential()
vision_model %>%
layer_conv_2d(filters = 64,
kernel_size = c(3, 3),
activation = 'relu',
padding = 'same',
input_shape = c(150, 150, 3)) %>%
layer_max_pooling_2d(pool_size = c(2, 2)) %>%
layer_flatten()
# tensor with the output of our vision model:
image_input <- layer_input(shape = c(150, 150, 3))
encoded_image <- image_input %>% vision_model
tabular_input <- layer_input(shape = 16, dtype = 'float32')
mlp_model <- tabular_input %>%
layer_dense(
units = 16,
kernel_initializer = "uniform",
activation = "relu")
# concatenate the metadata vector and the image vector then
# train a linear regression on it
output <- layer_concatenate(c(mlp_model, encoded_image)) %>%
layer_dense(units = 1, activation='linear')
# This is our final model:
vqa_model <- keras_model(inputs = c(tabular_input, image_input), outputs = output)
The Question is related to Create a mixed data generator (images,csv) in keras.
Thanks for any help!
