2

I'm trying to write a bash function named myrun, such that doing

myrun script.py

with a Python file:

#MYRUN:nohup python -u script.py &

import time
print 'Hello world'
time.sleep(2)
print 'Once again'

will run the script with the command specified in the first line of the file, just after #MYRUN:.

What should I insert in .bashrc to allow this? Here is what I have now:

myrun () {
[[ "$1" = "" ]] && echo "usage: myrun python_script.py" && return 0
<something with awk here or something else?>
}
Ed Morton
  • 188,023
  • 17
  • 78
  • 185
Basj
  • 41,386
  • 99
  • 383
  • 673
  • That sounds like an awful lot of work just to avoid writing a second script... – Ignacio Vazquez-Abrams Jan 16 '16 at 16:18
  • 1
    Hang on - you want to write a bash function that calls an awk script when called from python? Doesn't that sound a teeny bit like you might be going off the rails? – Ed Morton Jan 16 '16 at 16:31
  • @EdMorton: No, not called from Python. Called from bash with `myrun script.py`. – Basj Jan 16 '16 at 16:32
  • So it's a python script that calls awk when it's called from bash? That's not any better. – Ed Morton Jan 16 '16 at 16:34
  • @EdMorton: You didn't get the idea. In the perspective of the Python script, this line is only a comment. See the accepted answer to see how it works. – Basj Jan 16 '16 at 16:35
  • You're right - I have absolutely no idea what it is you're trying to do or why. Good luck! – Ed Morton Jan 16 '16 at 16:36
  • @Basj if you're writing the commands yourself, why not make them execute if you pass a certain command-line option to the python script itself? That way you avoid the mess of executable comments and the script is all in one file. `./stuff.py` would run the normal script, and you could make `./stuff.py --prepare` (or something) run the stuff you'd otherwise have put in comments. – Score_Under Jan 16 '16 at 16:50
  • @Score_Under: interesting indeed. But the command I would like to do is often "start the python script itself with nohup". I can't do this *after* having started script.py. – Basj Jan 16 '16 at 17:51
  • 1
    @Basj You can! As an example, if you don't see `--no-fork` on the command line, you could do: `subprocess.Popen(['nohup', sys.executable, sys.argv[0], '--no-fork'] + sys.argv[1:]); exit()`. (I'd advise you check that `--no-fork` works before following this example though, or you'll end up with a process that keeps spawning then dying). – Score_Under Jan 16 '16 at 18:01
  • I'm just as lost as Ed was as to *why* you would want to do this, but I strongly suspect that if you could explain your underlying intent, we could tell you a better way to achieve the same larger goal. – Charles Duffy Jan 16 '16 at 20:46
  • ...for instance, a self-`exec`ing script (that re-executes itself inside a nohup wrapper) is actually quite easy to implement, whether in shell or Python. – Charles Duffy Jan 16 '16 at 20:46
  • @CharlesDuffy the reason is simple: Very often my script needs to be run *not with* `./myscript.py` but with a *specific hard-to-remember* command line command : `nohup python blablabla > blabla &>2!>@ blabla &`. Of course I could do a `myscript.sh` script along the `.py` but I hate to have to deal with 2 files all the time. That's why I thought about this solution: *the command to run the script is in the first line of the .py* so that 1) single file solution 2) I'll always remember this specific command (it's in the .py file's first line, as a comment) 3) easy to run with `myrun script.py` – Basj Jan 16 '16 at 22:13
  • @Basj, but you can do all that from within the script itself! `nohup` doesn't actually do any magic you can't do natively in Python; you just need to reopen file descriptors 0, 1 and 2, and override the signal handler for HUP. If you wanted that to be default behavior, you could do as many other programs do and take those actions unless passed a command-line argument indicating that your program should stay in the foreground (`-F` or such). – Charles Duffy Jan 16 '16 at 22:16
  • Oh -- and to simulate the `&`, you need a double `fork()`. Still easy to do from Python, and I'm sure if you searched for "python daemonize self" you'd find plenty of samples showing you how. – Charles Duffy Jan 16 '16 at 22:22
  • @CharlesDuffy : `reopen file descriptors 0, 1 and 2, and override the signal handler for HUP` : a few hints on how to do this in a few lines? Then, it's true I could avoid totally nohup etc. The main thing I need is : I don't want the Python script to stop when I log out from SSH. That's the main thing I want. PS: If possible, I'd like to do it manually in a few lines, rather than requiring another 3rd party module – Basj Jan 16 '16 at 22:36
  • As I said, this is easy to search for. http://stackoverflow.com/a/1603152/14122 covers the double-fork part. `sys.stderr.close(); sys.stdout.close(); sys.stdin.close(); os.close(0); os.close(1); os.close(2)` is the other part -- though you might want to replace them sys.stdin with, say, `open('/dev/null', 'r')` and stdout and stderr with similar file objects pointing to `/dev/null` or a log. – Charles Duffy Jan 16 '16 at 23:22
  • Thanks @CharlesDuffy. Just being curious, why `double-fork`? [This answer](http://stackoverflow.com/a/1603152/1422096) uses a single fork whereas [this one](http://stackoverflow.com/a/6374881/1422096) needs two. Do you think the simple solution with only one will work? – Basj Jan 16 '16 at 23:27
  • Moreover, `os.dup2()` can be used to copy an arbitrary FD -- like one that `os.open()` gives you -- to a specific file descriptor number, so you can point them at `/dev/null` for use in the case that a C module or external program you run tries to write directly to the FDs rather than going through `sys.stdin` / `sys.stdout` / `sys.stderr`. – Charles Duffy Jan 16 '16 at 23:27
  • That's also covered here on StackOverflow. No reason to ask me directly. First link: http://stackoverflow.com/questions/881388/what-is-the-reason-for-performing-a-double-fork-when-creating-a-daemon – Charles Duffy Jan 16 '16 at 23:28

2 Answers2

1

This is unrelated to Bash. Unfortunately, the shebang line cannot portably contain more than a single argument or option group.

If your goal is to specify options to Python, the simplest thing is probably a simple sh wrapper:

#!/bin/sh
nohup python -u <<'____HERE' &
.... Your Python script here ...
____HERE
tripleee
  • 175,061
  • 34
  • 275
  • 318
  • I'd argue that even simpler than this would be to do `nohup`'s work in native Python; it's not that hard to reopen the standard FD set and set signal-handling behavior for HUP. – Charles Duffy Jan 16 '16 at 22:19
1

A minimalist version:

$ function myrun {
  [[ "$1" = "" ]] && echo "usage: myrun python_script.py" && return
  local cmd=$(head -n 1 < "$1" | sed s'/# *MYRUN://')
  $cmd
}

$ myrun script.py
appending output to nohup.out
$ cat nohup.out
Hello world
Once again 
$

(It's not clear to me whether you're better off using eval "$cmd" or simply $cmd in the last line of the function, but if you want to include the "&" in the MYCMD directive, then $cmd is simpler.)

With some basic checking:

function myrun {
  [[ "$1" = "" ]] && echo "usage: myrun python_script.py" && return
  local cmd=$(head -n 1 <"$1")
  if [[ $cmd =~ ^#MYRUN: ]] ; then cmd=${cmd#'#MYRUN:'}
  else echo "myrun: #MYRUN: header not found" >&2 ; false; return ; fi
  if [[ -z $cmd ]] ; then echo "myrun: no command specified" >&2 ; false; return; fi
  $cmd  # or eval "$cmd" if you prefer
}
peak
  • 105,803
  • 17
  • 152
  • 177
  • Thanks! Maybe we should add something to make $cmd empty if `#MYRUN:` is not found... Currently it would try to run the first line anyway! – Basj Jan 16 '16 at 16:37
  • 1
    If your command is specified with shell syntax, it may be cleaner to use `eval "$cmd"` rather than just `$cmd` (which has totally different rules for how the command is parsed). – Score_Under Jan 16 '16 at 16:46
  • I'd argue that the `eval` approach will always be more desirable. See BashFAQ #50 -- http://mywiki.wooledge.org/BashFAQ/050 -- for a discussion of the pitfalls in the other approach (quotes are treated as literal data, spaces can't possibly be escaped, etc). – Charles Duffy Jan 16 '16 at 21:17