Shell script child process interactive terminal session

Question

I am running a shell script which restarts my app when certain files change. However, when I run the app via a script, a call to enter the debugger no longer works, I think because the app is now running in the background, while the shell script is the foreground.

The debugger works just fine without this script. This all runs inside a Docker container, but I don’t think that’s relevant since the app is being run with docker-compose run which attaches a terminal.

The script, which I inherited in this codebase, is as follows:

#!/bin/sh
trap Terminate SIGINT SIGTERM SIGQUIT SIGKILL

Start () {
  echo "------- Entrypoint: starting app"

  bundle exec puma -C dev/puma.rb &

  echo $! > /tmp/entrypoint_process.pid
}

Stop () {
  echo "------- Entrypoint: stopping app"

  kill `cat /tmp/entrypoint_process.pid`
  rm /tmp/entrypoint_process.pid
}

Terminate () {
  Stop
  echo "------- Entrypoint: terminating"
  exit 0
}

Restart () {
  Stop
  Start $@
  WatchForChanges $@
}

WatchForChanges () {
  echo "------- Entrypoint: watching for changes"

  # Watch for files that are:
  # - Not handled by the Loader.
  # - Don’t require an external action (e.g. a change to the Gemfile needs `docker-compose run signup3 bundle install` to be run before it’s worth triggering a restart).
  #
  inotifywait -r -q -e modify \
    boot.rb \
    config.ru \
    Gemfile.lock \
    services/loader.rb

  Restart $@
}

WatchForChanges $@ & Start $@
wait

I believe the ampersand at the end of bundle exec puma -C dev/puma.rb & is the ‘problem’, because it runs the app in the background. But if I remove it the PID is not stored (because it blocks within Start ()) and so Stop fails, and then Start fails too because the app tries to bind to the same port as the already-running copy of itself.

Is there a reliable way to get this working?

Update:

Having removed the & along with echo $! > /tmp/entrypoint_process.pid, I modified Stop () to kill the app without a PID using this:

kill $(ps aux | grep '[p]uma' | awk '{print $2}')

(Based on this Q&A)

This works to some extent: if the app tries to drop into the debugger before this script triggers Restart () then it succeeds. However, once the app has been restarted by the script it again fails to drop into the debugger as before.

Is there something about the way the functions are being called that causes the app process to be put in the background again when the script runs?

I tried making the calls inside Restart () match the initial call to WatchForChanges $@ & Start $@, but that didn’t help.

Note that bare `$@` acts like `$*`, string-splitting and glob-expanding values. Use `"$@"`, _with the quotes_, to pass arguments through exactly as they're received. — Charles Duffy, Jul 01 '20 at 16:14

score 0 · Answer 1 · answered Jul 01 '20 at 16:11

0

I worked around this in the end by changing approach: instead of wrapping my app in a shell script as above, I call into the inotify API directly in another thread within my app, and that thread calls exit when it detects a change.

This is not as elegant as it could be, because it then relies on Docker’s restart-on-error functionality to start the app up again (which is very slightly slower than restarting inside Docker). But it works well enough to provide a decent experience when developing.

answered Jul 01 '20 at 16:11

Leo

4,217
4
25
41

Have you considered `incron`, instead of reinventing this wheel yourself? See http://inotify.aiken.cz/?section=incron&page=doc – Charles Duffy Jul 01 '20 at 16:15
No! I inherited this script and I didn’t know about `incron` – thanks. However, the core problem here is the obligation to run the app process in the background. I would imagine that a command run by `incron` would still be in the background, no? – Leo Jul 01 '20 at 16:18
If incron itself is in the background, something it runs will remain in the background (from the perspective of the controlling TTY, assuming that's the respect in which you're using the phrase "in the background"). – Charles Duffy Jul 01 '20 at 16:29
...that said, backgrounding tasks doesn't really fit with the ideal "one entrypoint-process per container, and everything else is a child of it" Docker model. – Charles Duffy Jul 01 '20 at 16:30
...personally, if I were designing a system to rebuild a service on a code change, I would probably use socket activation to route new connects to the very latest version, while letting old versions finish handling connections that were already made to them gracefully. See http://0pointer.de/blog/projects/socket-activated-containers.html – Charles Duffy Jul 01 '20 at 16:34
Right, gotcha. Yes that’s exactly what I mean – the TTY needs to be present in order for the debugger to stop when it hits a breakpoint. Otherwise it just prints some info and carries on. – Leo Jul 01 '20 at 16:40
I’ll take a look at the link you’ve sent about socket activation. This behaviour is _only_ for development environments though, so it’s nice if it’s fast, but it doesn’t have to be 100% glitch-free. – Leo Jul 01 '20 at 16:42
Ah – I think `systemd` may not be a straightforward option here – this is a fairly minimal Docker image based on a version of `ruby:alpine` – Leo Jul 01 '20 at 16:45
That is really interesting though that it can map host sockets to container sockets on demand as well. – Leo Jul 01 '20 at 16:47
Wasn't meaning running systemd in the container, but having the copy on your host do the work of forwarding sockets into containers. – Charles Duffy Jul 01 '20 at 16:59
...jumping back to debugging, though, you don't need your debugger to be running on the same machine as the process being debugged. Debugging over a socket is a thing, and has been for decades. Heck, back when I did embedded work, we did debugging over _serial lines_. I don't know the node.js stack, but I would be utterly shocked if they didn't have competent remote debugging support (how are developers supposed to use GUI tools to debug processes running on headless systems or cloud servers without GUI libraries otherwise?) – Charles Duffy Jul 01 '20 at 17:00
(ahh, `node --inspect` to enable remote debugging; then you just need to be able to forward a TCP connection in from wherever you actually want the debugger to run). – Charles Duffy Jul 01 '20 at 17:03
Ah okay – I think that might be the way to go here. Pry does have a remote mode, as you say, and then we don’t have to worry about having the main process attach to the TTY any more. – Leo Jul 01 '20 at 17:04

Shell script child process interactive terminal session

Update:

1 Answers1