0

I need to use multiple (my) TF CNN models for a rather complex task. I also need to use easyOCR (PyTorch) every now and then. However easyOCR usage is pretty rare and the task is very small in comparison to TF models inference. Therefore I use gpu=False in easyocr.Reader constructor. Nevertheless, as soon as easyocr predicts anything, GPU is allocated for for pyTorch (This is a known bug, I already checked easyOCR's github issues) and any TF model throws error:

Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so tr
y looking to see if a warning log message was printed above.

If I predict with any TF model first, the easyocr model throws:

RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/THCCachingHostAllocator.cpp:278

I have found a workaround, but it seems rather dangerous to put something like this into production.

Is there a more safe way of achieving this?

talonmies
  • 70,661
  • 34
  • 192
  • 269
Vlade
  • 89
  • 1
  • 7
  • 1
    How much GPU memory do your tensorflow models use up? If they stay below the limit you set in the solution you linked, that sort of code isn't dangerous. We have plenty of code like it running in our production environment. Just make sure the sum of all GPU usage is below the total GPU memory available – Skum May 27 '21 at 08:45
  • It varies - I use 7-8 different CNN models for the task. However most of them should be safe (or I can lower the batch size for inference a little bit). But this should be safe. I am not sure about the easyOCR's need for memory in a long run. If nothing better comes up I will be forced to use this solution. – Vlade May 27 '21 at 09:00

0 Answers0