0

I've got a process I'm trying to run within a chroot. Said chroot is entered via a python script that I didn't write (it's Chromium's cros_sdk script, if that helps anyone). Here's my code:

def start_devserver(image_folder):
    """
    Start a devserver, serving the image in image_folder
    """
    # path manipulation stuff goes here
    devserver_command = [sdk_path, "--", devserver_exe, "--pregenerate_update",
                     "--image={}".format(image_path)]
    server_process = subprocess.Popen(devserver_command,
                                      stdout=subprocess.PIPE,
                                      stderr=subprocess.STDOUT,
                                      cwd=os.path.expanduser("~/chromiumos"))
    return server_process

def standup_devserver(image_folder):
    """
    Start the devserver and monitor its output until the update has
    been generated and the local server is ready to accept connections
    """
    devserver_process = start_devserver(image_folder)
    devserver_update_pat = re.compile(r'cache/[a-z0-9]+/update.gz')
    devserver_update = None
    devserver_started = False
    print "monitoring devserver progress"
    while not devserver_started:
        output = devserver_process.stdout.readline()
        if output:
            print output
        if output is None:
            break
        if devserver_update is None:
            match = re.search(devserver_update_pat, output)
            if match is not None:
                devserver_update = os.path.join("/var", "lib",
                                                "devserver",
                                                "static",
                                                match.group())
            LOG.info("update generated and stored in %s",
                        devserver_update)
        else:
            if "ENGINE Bus STARTED" in output:
                print "devserver started"
                devserver_started = True
    else:
        return devserver_update, devserver_process


def main(image_folder):
    """ 
    Main entry point--stand up the devserver and collect the necessary files.
    Returns the path of the temp directory containing the files
    """
    update, process = standup_devserver(image_folder)
    print "tmp_dir: {}".format(update)
    tmp_dir = collect_files(update)
    process.terminate()
    return tmp_dir

Two very weird things are happening:

1) It's forking multiple processes which aren't killed when I do server_process.terminate():

 5708 pts/8    T      0:00 python
 5712 pts/8    T      0:00 /usr/bin/python2 /.../chromiumos/chromite/bin/cros_sdk -- /.../trunk/chroot/usr/lib/devserver/devserver.py --pregenerate_update --image=/.../trunk/src/build/images/.../chromiumos_base_image.bin
 5988 pts/8    S      0:00 /usr/bin/python2 /.../chromiumos/chromite/bin/cros_sdk -- /.../trunk/chroot/usr/lib/devserver/devserver.py --pregenerate_update --image=/.../trunk/src/build/images/.../chromiumos_base_image.bin
 5993 pts/8    S      0:00 /usr/bin/python2 /.../chromiumos/chromite/bin/cros_sdk -- /.../trunk/chroot/usr/lib/devserver/devserver.py --pregenerate_update --image=/.../trunk/src/build/images/.../chromiumos_base_image.bin
 6208 pts/8    Sl     0:09 /usr/bin/python2.7 /.../trunk/chroot/usr/lib/devserver/devserver.py --pregenerate_update --image=/.../trunk/src/build/images/.../chromiumos_base_image.bin
 8322 pts/8    S+     0:00 grep --color=auto python

(ellipses replace parts of my paths I'd rather not make public, but rest assured they're all the same in each process.)

2) When server_process is returned to the interpreter, the interpreter gets backgrounded. My main function calls a function that calls a function that calls the function that starts & returns the process, but I don't get dumped out into bash until main completes:

sam(~/PycharmProjects/autoupdates)||$ sudo python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from local_devserver import main
>>> td = main("latest")
<snip>
devserver started
tmp_dir: /var/lib/devserver/static/cache/ae38287dde7f3f17ce03f36942d789fa/update.gz
moving update from /home/sam/chromiumos/chroot/var/lib/devserver/static/cache/ae38287dde7f3f17ce03f36942d789fa/update.gz to /tmp/tmpAX0Bm2/update.gz

[1]+  Stopped                 sudo python

If anyone with a deeper understanding of subprocess than myself could offer some clarity, it'd be much appreciated.

swizzard
  • 1,037
  • 1
  • 12
  • 28
  • It sounds rather like the subprocess is doing something like `daemonize`, whose whole purpose is to get out from under the direct control of the parent process, and also may do odd things with the controlling terminal. If there's an option for "don't daemonize" or "run in foreground", that might be tried. – Tom Hunt Feb 27 '15 at 20:26
  • does the subprocess `fork()` any more child processes ? – cmidi Feb 27 '15 at 20:37
  • 1
    What does "get backgrounded" mean? What do you see? What do you expect to happen instead? Is it really necessary to include all these path manipulations in the code example while the code that consumes the output of the subprocess and waits for it to finish is not shown? – jfs Feb 27 '15 at 20:38
  • It is normal that `.terminate()` doesn't kill children. You could [try to send a signal to the whole process group if children don't change it](http://stackoverflow.com/q/4789837/4279). Try to run via `strace` to find out what causes SIGTSTP signal. – jfs Mar 02 '15 at 18:52

0 Answers0