I am running a python program that has a large CPU memory consumption. I have limited real CPU memory (32G), but the swap memory is large enough (128G). Can I limit the python program to use less real memory (like up to 4G) and it turns to swap for more?
I am working on Ubuntu 20.04 with python 3.8. Thanks!
If you've to ask this question, then you're (still) lacking the knowledge that it'd take to make sensible choices about where to place memory.
What you must understand is, that on Linux all memory not directly allocated from dedicated pools, set up manually, will essentially but just cut from the swap file / device I/O cache. The figure important to you is, how much of that memory can be locked so that it won't get swapped out eventually (memory lock limits; can be set per user, and then reduced for each process).
Also keep in mind that whenever you're accessing some data that's been swapped out, it first had to be read back into RAM, in order to be workable.
If your concern is, that your program might hog up all the available memory, leaving the system unusable, then reduce it's memory lock limit (man 1 prlimit). Or you might consider running it under a user that had it's limit fixed in an unchangeable way. (man 5 limits.conf).
