0

I am trying to train a model with my own data by using GPU. There are 8 classes of objects to be detected, however when I start the training I am getting an allocation error.

I have tried to use different pre-trained models, but it continued to give me the same error. I have also tried to reduce the "batch-size" in the .config file, but it only worked when I equal the batch-size to 1.

I've followed this link to train a model with custom data: https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html#training-the-model

Is my hardware not enough or something else?

GPU: Nvidia GTX 1650

CPU: Intel i7 9750H

Memory: 16 GB

batuhancan
  • 37
  • 5
  • GPU's are really good at tensorflow because of the way it is built. I can't tell you specifically how much power you need, but I have made models on much less (albeit without using tensorflow). If you are having problems, try going off of your GPU, because it looks a lot beefier to tensorflow. – coderman1234 Mar 05 '21 at 14:11
  • Yeah that is more than enough power to train a model. If you are just getting started, fast.ai is probably a really good place to start. You can train a world class model in minutes. – Chris Mar 05 '21 at 14:13
  • You can try using the concept of gradient accumulation that is a clever trick to virtually use a larger batch size without too much GPU memory. You can read more about it in [this](https://stackoverflow.com/questions/62067400/understanding-accumulated-gradients-in-pytorch) post – Jitesh Malipeddi Mar 18 '21 at 17:00

1 Answers1

1

Well your hardware certainly seems impressive, I doubt there'll be a problem with that. What you can do is probably look at other sites, this is what I found:

1.https://www.tensorflow.org/tutorials/customization/custom_training_walkthrough 2.https://towardsdatascience.com/custom-object-detection-using-tensorflow-from-scratch-e61da2e10087

Aryan Arora
  • 46
  • 3