Is it possible to get the PID of the sibling process by using `psutil` or any other Python packages?

Question

I am running a set of parallel computing.

I am trying to use psutil to track the computation (If someone has a better solution, plz tell me)

>>> p = psutil.Process(4370)
>>> p.cpu_percent()
0.0
>>> p.cpu_times()
pcputimes(user=6440.78, system=5.4, children_user=0.0, children_system=0.0)
>>> p.cpu_affinity()
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
>>> p.cpu_num()
2

I guess the last one get a value of 2 for p.cpu_num() indicates that this job is on parallel computing, there is another sibling process doing the computing simultaneously.

Is it possible to get the PID of the sibling process by using psutil or any other Python packages?

did you try to get `.parent()` and later use `.process_iter()` to get all procces and check which one has the same parent ? — furas, Sep 10 '19 at 03:11
simply track the computation within the computation itself, not outside. — Massimo Cafaro, Sep 10 '19 at 14:11

score 0 · Accepted Answer · answered Sep 10 '19 at 19:50

"I guess the last one get a value of 2 for p.cpu_num() indicates that this job is on parallel computing"

No, this does not mean anything other but that a process p is currently mapped onto the second CPU from all CPU-s available ( O/S task scheduler decides on which CPU/core a job is going to get executed + the process-affinity settings may restrict such a choice )

>>> print( p.cpu_num.__doc__ )
Return what CPU this process is currently running on.
            The returned number should be <= psutil.cpu_count()
            and <= len(psutil.cpu_percent(percpu=True)).
            It may be used in conjunction with
            psutil.cpu_percent(percpu=True) to observe the system
            workload distributed across CPUs.

Q : Is it possible to get the PID of the sibling process by using psutil

Yes, it is. How? It is quite enough to follow the documentation to assemble any traversing tree strategy that meets your needs and expectations upto a system-level universal process-monitor.

>>> aParentOfThisPROCESS = psutil.Process( thisProcess.parent().pid )
>>> aParentOfThisPROCESS.threads()
6
>>> aParentOfThisPROCESS.open_files()
[popenfile(path='/XXXXXXXXXXXXXXXX', fd=8, position=0, mode='r', flags=32768)]

>>> print( aParentOfThisPROCESS.children.__doc__ )
Return the children of this process as a list of Process
        instances, pre-emptively checking whether PID has been reused.
        If *recursive* is True return all the parent descendants.

        Example (A == this process):

         A ─┐
            │
            ├─ B (child) ─┐
            │             └─ X (grandchild) ─┐
            │                                └─ Y (great grandchild)
            ├─ C (child)
            └─ D (child)

        >>> import psutil
        >>> p = psutil.Process()
        >>> p.children()
        B, C, D
        >>> p.children(recursive=True)
        B, X, Y, C, D

        Note that in the example above if process X disappears
        process Y won't be listed as the reference to process A
        is lost.

Thanks for your answer. "a process p is currently mapped onto the second CPU", is the O/S task scheduler a part of Python or Operating System? — , Sep 10 '19 at 22:37
O/S is an abbreviation of **O**perating **S**ystem. O/S is a system, that supervises how the hardware is being used, operates the task scheduler (which process/thread gets a permit to load onto real CPU and have some amount of time to get executed a part of the code, before it gets signaled and been evicted from CPU to a rear-side of the task-scheduler queue, and some other process/thread gets loaded in that place ). This is how O/S-es work. Even if any application ( be it python, web-browser, whatever ) asks for 120-threads, the O/S scheduler decides what/when/where/for how long gets exec'd — user3666197, Sep 10 '19 at 22:44
Consider this [example](https://stackoverflow.com/q/57865808/10449636), running parallel on Ubuntu 18.04 with Python 3.7.3, numpy 1.16.4, sklearn 0.21.2, is there another intermediate scheduler/wrapper between the task and OS scheduler, such something in joblib package or any other Python parallel package? — , Sep 11 '19 at 00:59

Is it possible to get the PID of the sibling process by using `psutil` or any other Python packages?

1 Answers1