3

I can create a subprocess in Python and suspend it like this:

proc = subprocess.pOpen(['binary', 'arg1', 'arg2'])
# Subprocess was created in a running state
proc_handle = psutil.Process(pid=proc.pid)
proc_handle.suspend()
# Subprocess is now suspended
proc_handle.resume()
# Subprocess is now running

Question: How can I start the subprocess in a suspended state? I understand that very little time will elapse between creating the subprocess and suspending it, but I am interested in a seamless solution. Also, I know I could easily wrap the subprocess in a parent process which will wait for a resume signal before proceeding, but I would like a less hacky, more elegant solution if one exists. Furthermore, I know I could create a class wrapper around the subprocess, that won't create it until I actually want to run it, but then I won't have a PID assigned to it yet and I won't be able to check stats like memory footprint of the unstarted process.

I would love it if I could do this:

cmd = ['binary', 'arg1', 'arg2']
proc = subprocess.pOpen(cmd, autostart=False) # autostart argument does not exist
# Subprocess was created, so it had a PID and has a memory footprint, 
# but is suspended and no code had been run

I recognized while I was writing the above example of pOpen with autostart=False that this may not be possible because of how processes are run. If I want to break at the start of the main() function, then some code will have already been executed as part of static initialization. Ideally, the subprocess will be suspended before that step, just after the process memory is created and initialized.

Note: I am particularly interested in solving this issue in a linux environment but a platform-independent solution would be great if it exists.

JimPri
  • 1,278
  • 8
  • 17
  • Can you add in pseudo code what you are trying to achieve? – TwistedSim Apr 24 '18 at 13:28
  • 2
    What is the larger problem you're trying to solve? Because it sounds like you should just create and start the subprocess at a later point in your code, where it now has `proc_handle.resume()`. –  Apr 24 '18 at 13:31
  • 3
    I am not aware of a way of doing this *on the command line / shell directly* in Linux. Given the answers to [this question](https://serverfault.com/questions/293632/how-do-i-start-a-process-in-suspended-state-under-linux), it seems the standard way of doing this is sending `SIGSTOP` after the process has started, which is effectively what you already do. There are suggested workaround in the answers, perhaps that can help you. But consider first if you really need this (see my other comment). –  Apr 24 '18 at 13:33
  • @Evert I saw that question too, which makes me think that this is impossible without enlisting the help of a debugger which actually instruments the code. My only argument against deferring the creation until I want the subprocess to actually run is that then I don't have access to the PID and I can't check stats like the memory footprint at creation. – JimPri Apr 24 '18 at 13:36
  • And, no, I don't __really__ need this functionality to achieve my desired outcome. I just started thinking about it when I was working on this and wanted to know if there is a standard way to do it. Looks like the answer is no. – JimPri Apr 24 '18 at 13:39
  • What exactly do you mean by 'memory footprint at creation' and why/what do you need it for? This might be helpful info: https://unix.stackexchange.com/a/175409/282757 – Tom Dalton Apr 24 '18 at 13:43
  • 1
    If it's "unstarted", there hasn't been an `execve()` yet, so the memory for it hasn't been allocated. Or if you stop *right* at the execv syscall, then the loader hasn't yet run. It sounds like you're wanting something extremely particular (and yet not specifying same). – Charles Duffy Apr 24 '18 at 13:45
  • 1
    Which is to say -- you could certainly cause your program to stop itself from the child end after the fork and before the exec (so after there's a new PID allocated, but before the program running associated with it becomes your `binary` and not a copy of the Python interpreter), but I doubt that that's actually helpful towards your intended goal. – Charles Duffy Apr 24 '18 at 13:47
  • ...what you *might* consider (not Python-specific at all, but as a mechanism to interrupt the binary loader at a point of your choice) is building an auditor and pointing at it with the `LD_AUDIT` environment variable; see [`man 7 rtld-audit`](http://man7.org/linux/man-pages/man7/rtld-audit.7.html). Hooking at `la_preinit()` may be what you want. – Charles Duffy Apr 24 '18 at 13:51

1 Answers1

3

Stopping After The fork(), Before The exec()

This one's easy, though not so useful:

import subprocess, os, signal
proc = subprocess.Popen(
    ['binary', 'arg1', 'arg2'],
    preexec_fn=lambda: os.kill(os.getpid(), signal.SIGSTOP),
)

Why "not so useful"? Because while preexec_fn runs in the new PID, your intent appears to involve gathering statistics on memory usage (to the extent that that can be done before a program does its malloc() calls), and until the exec has taken place, the program whose memory you'll be measuring is the Python interpreter.

Stopping After The exec(), Before Invocation Of main()

First, compile an audit module for the Linux loader (ld.so), akin to the following:

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <signal.h>

unsigned int la_version(unsigned int version) {
    return version;
}

/* To have stronger guarantees of immediacy, I might replace this with flock()ing a
 * lockfile, and releasing that lock from the parent when ready to continue. YMMV,
 * consider this a stub intended to be replaced.
 */
void la_preinit(uintptr_t *cookie) {
    kill(getpid(), SIGSTOP);
    unsetenv("LD_AUDIT");        /* avoid our children getting stopped the same way */
}

Then, specify it when running your subprocess:

newEnv = dict(os.environ)
newEnv['LD_AUDIT'] = '/path/to/above/library.so'
proc = subprocess.Popen(['binary', 'arg1', 'arg2'], env=newEnv)

la_preinit is invoked by the loader just before main() is started.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441