3

I am training a unet model. I started training with my pc, but it was too slow as it didn't run with GPU. Therefore I started using Google colab for faster training (using GPU). I have the same dataset saved locally and in Google Drive and I also have the exact same code in colab and in my pc, except paths as I need to change them to read from my Google Drive files.

My problem is that the results of training the unet in my computer differ a lot from those obtained training the unet with Google colab.

I don't know why this happens (I use the same seed in both and I have tested that I always obtain the same results locally if I use that seed). When I train the unet in my pc I obtain more or less 90% accuracy. However, when I train it using colab with GPU I only obtain 65%. I've also tried to use CPU in colab and I get the same issue.

Thanks

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
IVR
  • 43
  • 1
  • 6

2 Answers2

1

I also had this problem: Wild discrepancies between training DeepLab ResNet V3 on Google Colab versus on local machine

This will solve your accuracy issues, however this means that it will take ages to train since you're reading the data from Google Drive and not from your Colab workspace. This fix resulted in worse performances than on my local machine though...

James Heaton
  • 61
  • 4
0

I also went through the exact same problem. For me the problem was with random split into train and test, It was just a coincidence that random split on colab got the good performance while bad on the local machine. Then just using random_state fixed the issues for me. E.g.: train_test_split(data, target, test_size=0.1, random_state=30, stratify=target)

Gulshan Arya
  • 1
  • 1