How Can I Run A Command That Does Not Exit Upon Failure Until It Succeeds?

Question

I have a Docker container running HBase See here for more details about it. Basically, it takes around 30s for HBase to come alive.

Obviously I have other containers depending on this one. In one of them, in docker-entrypoint.sh I am basically doing this:

#!/bin/bash

/path/to/command argument

while [ $? -ne 0 ]; do
  sleep 2;
  /path/to/command argument
done

The problem here is that the /path/to/command is something that deploys a webserver, that would interact with HBase container. If it succeeds, it is blocking the current process (which is fine, since it is in a container), but in a same way, if it fails, it prints the stack trace but does not exit with some -ne 0 code (Actually, does not exit at all). In fact, if it exits --restart=always would handle it.

So, my question here is that how can I keep it retrying until it succeeds, basically without exiting with some fail code? Exiting from the Docker container is also an option, so that it can keep restarting until success. For now, I do not have the opportunity to modify the not exiting code to exit.

Update:

I had to do like this:

#!/bin/bash

tmp=$(mktemp)
/path/to/command args 2> "$tmp" &

while true; do
  sleep 5;
  if [ -s "$tmp" ]; then
    exit
  fi
done

Basically, I had to run it in the background so that the script above would continue executing (path/to/command is a UI blocking command -such as listen of a server - that does not exit when succeeds -it listens- nor when it fails -it should have, this is why I am asking this question-).

So in order for Docker container not to exit , I had to add a while true; loop, in which I am checking if there is an error in$tmp. If there is, I simply exit.

Update 2:

I changed the bash script to:

#!/bin/bash

/path/to/command args &

while true; do
  sleep 10;
  
  response=$(curl --write-out %{http_code} --silent --output /dev/null http://localhost);

  if [ "$response" != "200" ]; then
    exit
  fi
  
done

since my container is deploying a web server, I am at least monitoring its status in the main thread, after starting it. Still there's room for improvement.

Side note: [This](http://stackoverflow.com/questions/5274294/how-can-you-run-a-command-in-bash-over-until-success) and a few others asks the same question for an exiting process. — Hasan Can Saral, Jun 08 '16 at 09:26
Might be able to redirect stderr to a file, then check if said file is not empty. — Andreas Louv, Jun 08 '16 at 09:30
You could also run it as a background process and join with an alarm... — Will, Jun 08 '16 at 09:43
@Will, I can run it as a background process, but didn't quite get/know what you meant by joining with an alarm. — Hasan Can Saral, Jun 08 '16 at 09:45

score 2 · Accepted Answer · answered Jun 08 '16 at 09:46

2

You could try redirecting stderr to a file, and then check if said file is empty:

tmp=$(mktemp)
/path/to/command argument 2> "$tmp"

# Run code until "$tmp" is empty
until [ -s "$tmp" ]; do
  sleep 2
  /path/to/command argument 2> "$tmp"
done

rm "$tmp"

answered Jun 08 '16 at 09:46

Andreas Louv

46,145
13
104
123

Should I not clear `$tmp` when I am going to run the command again? – Hasan Can Saral Jun 08 '16 at 09:48
1

@HasanCanSaral `2>` should do that for you. Consider `echo 1 > a;echo 2> a; od a;` will output `0000000` meaning it's empty – Andreas Louv Jun 08 '16 at 09:48
A quick question, will I end up with a couple of stuck processes, and one running upon success? How could I manage to exit from `Docker`container when the `$tmp` is not empty, which I believe is a better option? – Hasan Can Saral Jun 08 '16 at 09:56
@HasanCanSaral Your processes will terminate when they print the stack trace, if they don't how would your code continue to execute? And the `until` loop will end when the tmp file is not empty, then remove the file. And exit. – Andreas Louv Jun 08 '16 at 10:34
The problem is they don't. They print the stack trace and hung, which is why I am asking this question. If they were to terminate, I would not have this issue, since Docker containers would exit then restart. I had to do it like above (see the edited post), but am not quite happy about it. – Hasan Can Saral Jun 08 '16 at 20:26
@HasanCanSaral I can see it's not a pretty solution. I will inform you if I come up with something more elegant. – Andreas Louv Jun 08 '16 at 20:40
Thank you, greatly appreciated! – Hasan Can Saral Jun 08 '16 at 20:42
I came up with my second update, in the original post. A little better, since I am checking the status of the web server, but there's still room for improvement, I guess. – Hasan Can Saral Jun 14 '16 at 13:03

How Can I Run A Command That Does Not Exit Upon Failure Until It Succeeds?

Update:

Update 2:

1 Answers1