0

I have a cloud dataset with 12 frames per hour (one every 5 minutes). I'm considering approaching this every 12-frame as a video and analyzing cloud motion using 3D CNN. I want it to output a numeric value, which is an estimate of the future cloudiness of the region. So it's a "regression" problem. For this purpose, as you can see in my model below, I used the built-in function of Keras called Conv3D as the 3D CNN structure and added 1 Dense layer as the last layer (for regression), which gives an output.

def CNN3d(width=300, height=300, depth=12):
                                                                                                                              
   inputs1 = keras.Input((width, height, depth, 1))

   x1 = layers.Conv3D(filters=64, kernel_size=2, activation="relu")(inputs1)
   x1 = layers.MaxPool3D(pool_size=2)(x1)
   x1 = layers.BatchNormalization()(x1)

   x1 = layers.Conv3D(filters=64, kernel_size=2, activation="relu")(x1)
   x1 = layers.MaxPool3D(pool_size=2)(x1)
   x1 = layers.BatchNormalization()(x1)

   x1 = layers.GlobalAveragePooling3D()(x1)

   x1 = layers.Dense(units=128, activation="relu")(x1)
   x1 = layers.Dropout(0.3)(x1) 

   outputs = layers.Dense(units=1, activation="linear")(x1)

   model1 = keras.Model(inputs=[inputs1], outputs=[x1])
   return model1

model1 = CNN3d(width=300, height=300, depth=12)
model1.summary()

I had trouble in two parts:

1- Creating the data set.

So I tried to create train and test folders under a main folder. I thought of adding cloudiness images consisting of 12 frames in each child folder. i.e.:

train folder->1st video folder->12 frames of 1st video
             ->2nd video folder->12 frames of 2nd video
             ->3rd video folder->12 frames of 3rd video
             .
             . 

test folder->1st video folder->12 frames of 1st video
            ->2nd video folder->12 frames of 2nd video
            ->3rd video folder->12 frames of 3rd video
            .
            .

But since I'm not going to do classification, this may not be the logical way.

2- The part of labelling the data.

After creating the dataset, I was also sceptical about how I should label this data. Even if I put the labels in a csv file, how can CNN find out which 12-fold folder these labels belong to while training?

I wrote the following code to specify the train and test folders in my code:

training_set = train_datagen.flow_from_directory('/content/allimages/squareimages/train',
                                                 target_size = (300,300),
                                                 batch_size = 1,
                                                 class_mode = 'categorical')

test_set = test_datagen.flow_from_directory('/content/allimages/squareimages/test',
                                                 target_size = (300,300),
                                                 batch_size=1,
                                                 class_mode = 'categorical')

I'm quite sure I shouldn't have used "categorical" here, but couldn't find any better option

I have encountered the following error:

INVALID_ARGUMENT: required broadcastable shapes
[[{{node mean_squared_error/SquaredDifference}}]]
[[assert_greater_equal_1/Assert/AssertGuard/pivot_f/_23/_73]

I couldn't find any regression examples using 3DCNN. I leave the Colab link below so you can review my entire code:

https://colab.research.google.com/drive/1ocHFXosOVY7bSRWI99B14altb1LrQhGo?usp=sharing

And you can also reach out to the dataset I have used via the following link:

https://drive.google.com/file/d/1xYIPIZhexC4ZMqe7HnFBt6Oy1Lxk-Y79/view?usp=sharing

Thank you very much in advance.

  • Why don't you use [tf.keras.utils.load_img](https://www.tensorflow.org/api_docs/python/tf/keras/utils/load_img) to load the images yourself (without `flow_from_directory`)? You could store the file names and cloudiness values in a `csv` file – AndrzejO Sep 25 '22 at 22:35
  • @AndrzejO Hello, I have tried it but I think I should differ some other parts as well. I’m not sure if 3D CNN can sense the depth(12 frames) of my data. If you have a little time, could you also try it with my data link and my code link? I couldn’t accomplish this. – Emirhan Bilgiç Sep 25 '22 at 23:00
  • @AndrzejO and `flow_from_directory` is more practical because it's not a single image it's an image dataset. – Emirhan Bilgiç Sep 26 '22 at 07:02
  • I am curious, how do you decide about the 'cloudiness' of an image? Do you have this data? – AndrzejO Sep 26 '22 at 07:20
  • @AndrzejO yes I do, the dataset is from EUMETSAT. – Emirhan Bilgiç Sep 26 '22 at 07:21

0 Answers0