4

I have a single NVIDIA GPU which has a memory of 16GB. I have to run two different (and independent; meaning, two different problems: one is a vision type task, another is NLP task) Python programs. The codes are written using PyTorch and both the codes can use GPU.

I have tested that program 1 takes roughly 5GB of GPU memory, and the rest is free. If I run the two programs, will it hamper the model performance or will it cause any process conflicts?

Linked question; but it does not necessarily mean PyTorch codes

Coder
  • 151
  • 6

2 Answers2

1

I do not know the details of how this works, but I can tell from experience that both programs will run well (as long as they do not need more than 16GB of RAM when combined), and execution times should stay roughly the same.

However, computer vision usually requires a lot of IO (mostly reading images), if the other task needs to read files too, this part may become slower than when running both programs individually.

Joseph Budin
  • 1,299
  • 1
  • 11
  • 28
1

It should work fine.

In one of my projects, I faced the problem of lack of GPU memory while working with multiple models. After loading them, my models used to take up most of the GPU memory. And during model inference, very less memory used to remain for the data. As we know, if your models are loaded on GPU then you also need to load your data on your GPU. So when you do batch inference (for eg giving 16 images at a time to the model) the complete batch is loaded on the GPU. This again takes more GPU memory. Your program crashes if it does not get enough GPU memory.

If you think GPU memory is not the issue in your case then everything should work fine. You also do not need to worry about conflicts because both processes will allocate their own GPU memory and will work independently. There would be no performance issues.

Devashish Prasad
  • 1,227
  • 1
  • 13
  • 25