How to start a background process with nohup using Fabric?

Question

Through Fabric, I am trying to start a celerycam process using the below nohup command. Unfortunately, nothing is happening. Manually using the same command, I could start the process but not through Fabric. Any advice on how can I solve this?

def start_celerycam():
    '''Start celerycam daemon'''
    with cd(env.project_dir):
        virtualenv('nohup bash -c "python manage.py celerycam --logfile=%scelerycam.log --pidfile=%scelerycam.pid &> %scelerycam.nohup &> %scelerycam.err" &' % (env.celery_log_dir,env.celery_log_dir,env.celery_log_dir,env.celery_log_dir))

        

Answer 1

I'm using Erich Heine's suggestion to use 'dtach' and it's working pretty well for me:

def runbg(cmd, sockname="dtach"):
    return run('dtach -n `mktemp -u /tmp/%s.XXXX` %s' % (sockname, cmd))

This was found here.

Answer 2

As I have experimented, the solution is a combination of two factors:

• run process as a daemon: nohup ./command &> /dev/null &
• use pty=False for fabric run

So, your function should look like this:

def background_run(command):
    command = 'nohup %s &> /dev/null &' % command
    run(command, pty=False)

And you can launch it with:

execute(background_run, your_command)

Answer 3

This is an instance of this issue. Background processes will be killed when the command ends. Unfortunately on CentOS 6 doesn't support pty-less sudo commands.

The final entry in the issue mentions using sudo('set -m; service servicename start'). This turns on Job Control and therefore background processes are put in their own process group. As a result they are not terminated when the command ends.

For even more information see this link.

Answer 4

you just need to run

run("(nohup yourcommand >& /dev/null < /dev/null &) && sleep 1")

Answer 5

DTACH is the way to go. It's a software you need to install like a lite version of screen. This is a better version of the "dtach"-method found above, it will install dtach if necessary. It's to be found here where you can also learn how to get the output of the process which is running in the background:

from fabric.api import run
from fabric.api import sudo
from fabric.contrib.files import exists


def run_bg(cmd, before=None, sockname="dtach", use_sudo=False):
    """Run a command in the background using dtach

    :param cmd: The command to run
    :param output_file: The file to send all of the output to.
    :param before: The command to run before the dtach. E.g. exporting
                   environment variable
    :param sockname: The socket name to use for the temp file
    :param use_sudo: Whether or not to use sudo
    """
    if not exists("/usr/bin/dtach"):
        sudo("apt-get install dtach")
    if before:
        cmd = "{}; dtach -n `mktemp -u /tmp/{}.XXXX` {}".format(
            before, sockname, cmd)
    else:
        cmd = "dtach -n `mktemp -u /tmp/{}.XXXX` {}".format(sockname, cmd)
    if use_sudo:
        return sudo(cmd)
    else:
        return run(cmd)

May this help you, like it helped me to run omxplayer via fabric on a remote rasberry pi!

Answer 6

You can use :

run('nohup /home/ubuntu/spider/bin/python3 /home/ubuntu/spider/Desktop/baidu_index/baidu_index.py > /home/ubuntu/spider/Desktop/baidu_index/baidu_index.py.log 2>&1 &', pty=False)

Answer 7

nohup did not work for me and I did not have tmux or dtach installed on all the boxes I wanted to use this on so I ended up using screen like so:

run("screen -d -m bash -c '{}'".format(command), pty=False)

This tells screen to start a bash shell in a detached terminal that runs your command

Answer 8

This worked for me:

sudo('python %s/manage.py celerycam --detach --pidfile=celerycam.pid' % siteDir)

Edit: I had to make sure the pid file was removed first so this was the full code:

# Create new celerycam
sudo('rm celerycam.pid', warn_only=True)
sudo('python %s/manage.py celerycam --detach --pidfile=celerycam.pid' % siteDir)

Answer 9

You could be running into this issue

Try adding 'pty=False' to the sudo command (I assume virtualenv is calling sudo or run somewhere?)

Answer 10

I was able to circumvent this issue by running nohup ... & over ssh in a separate local shell script. In fabfile.py:

@task
def startup():
    local('./do-stuff-in-background.sh {0}'.format(env.host))

and in do-stuff-in-background.sh:

#!/bin/sh

set -e
set -o nounset

HOST=$1

ssh $HOST -T << HERE
   nohup df -h 1>>~/df.log 2>>~/df.err &
HERE

Of course, you could also pass in the command and standard output / error log files as arguments to make this script more generally useful.

(In my case, I didn't have admin rights to install dtach, and neither screen -d -m nor pty=False / sleep 1 worked properly for me. YMMV, especially as I have no idea why this works...)