0

I want to train a CNN via transfer learning and EfficientNetB3. I have a very large dataset of 512x512 images (I cannot reduce them) and whenever I try to train using batch_size larger than 32 my process doesn't even start training, it shows this for minutes and doesn't do anything else:

Epoch 1/10
2022-06-28 19:35:16.968212: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.

When I try to stop it my jupyter kernel times out, and sometimes a warning that my apps are running out of memory is shown. I'm using tensorflow-macos and tensorflow-metal.

I have tried using different generators for loading the dataset (custom and now tf.keras.utils.Sequential), so I'm sure that I'm NOT loading all the data at once, but it still doesn't work. My CNN has this architecture (I have tried some variations):

Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 rescaling_1 (Rescaling)     (None, 512, 512, 3)       0         
                                                                 
 efficientnet-b3 (Functional  (None, 16, 16, 1536)     10783528  
 )                                                               
                                                                 
 global_average_pooling2d_2   (None, 1536)             0         
 (GlobalAveragePooling2D)                                        
                                                                 
 dense_2 (Dense)             (None, 1)                 1537      
                                                                 
=================================================================
Total params: 10,785,065
Trainable params: 10,697,769
Non-trainable params: 87,296
_________________________________________________________________

I want to try larger batch sizes to get better results (it doesn't even work with 64) and I don't know what to do, and I haven't found much info on what can I do to solve this. I am using a M1 Mac Air with 16GB RAM, and I read that it has a good ML performance, so I don't know if the laptop is the problem. Is there any tensorflow configuration that I could try? I am not experienced in training neural networks so maybe I am missing something important. Should I just use Google Colab or something like that?

Shenia
  • 1
  • Well to do this you need more RAM, and I hate to be the bearer of bad news, but ARM Macs are not the best in terms of amount of RAM, 16 GB is not much considering it is shared with the CPU and GPU. – Dr. Snoopy Jun 28 '22 at 19:41
  • Does this answer your question? [How to prevent tensorflow from allocating the totality of a GPU memory?](https://stackoverflow.com/questions/34199233/how-to-prevent-tensorflow-from-allocating-the-totality-of-a-gpu-memory) – Djinn Jun 28 '22 at 21:59
  • @Djinn That does not apply to Apple's Tensorflow as memory is handled differently (due to unified memory). – Dr. Snoopy Jun 28 '22 at 22:59

0 Answers0