I have a cloud dataset with 12 frames per hour (one every 5 minutes). I'm considering approaching this every 12-frame as a video and analyzing cloud motion using 3D CNN. I want it to output a numeric value, which is an estimate of the future cloudiness of the region. So it's a "regression" problem. For this purpose, as you can see in my model below, I used the built-in function of Keras called Conv3D as the 3D CNN structure and added 1 Dense layer as the last layer (for regression), which gives an output.
def CNN3d(width=300, height=300, depth=12):
inputs1 = keras.Input((width, height, depth, 1))
x1 = layers.Conv3D(filters=64, kernel_size=2, activation="relu")(inputs1)
x1 = layers.MaxPool3D(pool_size=2)(x1)
x1 = layers.BatchNormalization()(x1)
x1 = layers.Conv3D(filters=64, kernel_size=2, activation="relu")(x1)
x1 = layers.MaxPool3D(pool_size=2)(x1)
x1 = layers.BatchNormalization()(x1)
x1 = layers.GlobalAveragePooling3D()(x1)
x1 = layers.Dense(units=128, activation="relu")(x1)
x1 = layers.Dropout(0.3)(x1)
outputs = layers.Dense(units=1, activation="linear")(x1)
model1 = keras.Model(inputs=[inputs1], outputs=[x1])
return model1
model1 = CNN3d(width=300, height=300, depth=12)
model1.summary()
I had trouble in two parts:
1- Creating the data set.
So I tried to create train and test folders under a main folder. I thought of adding cloudiness images consisting of 12 frames in each child folder. i.e.:
train folder->1st video folder->12 frames of 1st video
->2nd video folder->12 frames of 2nd video
->3rd video folder->12 frames of 3rd video
.
.
test folder->1st video folder->12 frames of 1st video
->2nd video folder->12 frames of 2nd video
->3rd video folder->12 frames of 3rd video
.
.
But since I'm not going to do classification, this may not be the logical way.
2- The part of labelling the data.
After creating the dataset, I was also sceptical about how I should label this data. Even if I put the labels in a csv file, how can CNN find out which 12-fold folder these labels belong to while training?
I wrote the following code to specify the train and test folders in my code:
training_set = train_datagen.flow_from_directory('/content/allimages/squareimages/train',
target_size = (300,300),
batch_size = 1,
class_mode = 'categorical')
test_set = test_datagen.flow_from_directory('/content/allimages/squareimages/test',
target_size = (300,300),
batch_size=1,
class_mode = 'categorical')
I'm quite sure I shouldn't have used "categorical" here, but couldn't find any better option
I have encountered the following error:
INVALID_ARGUMENT: required broadcastable shapes
[[{{node mean_squared_error/SquaredDifference}}]]
[[assert_greater_equal_1/Assert/AssertGuard/pivot_f/_23/_73]
I couldn't find any regression examples using 3DCNN. I leave the Colab link below so you can review my entire code:
https://colab.research.google.com/drive/1ocHFXosOVY7bSRWI99B14altb1LrQhGo?usp=sharing
And you can also reach out to the dataset I have used via the following link:
https://drive.google.com/file/d/1xYIPIZhexC4ZMqe7HnFBt6Oy1Lxk-Y79/view?usp=sharing
Thank you very much in advance.