1

I'm trying to run a program in docker, and once the program is successfully started, it creates a FIFO file in docker's file system, and writes a "success" string in it. I'm aware that if the file exists, I can stream the content of file by tail -f, but this will always wait until I hit ctrl-c on cli. And also, how to extend this case if the file has not been created yet?

I want to know if there's a shell command that I can wait till a file is being written non-empty string, and this file could have not existed while I start the wait. The wait will exit once it reaches a timeout.

Note that this command will be passed to a docker with docker exec -i myContainer the_desired_command....

Ugty
  • 73
  • 4
  • 1
    `tail -F` might be useful. – Benjamin W. Nov 22 '21 at 01:31
  • @BenjaminW. Is there a way to timeout a `tail -F` ? – Fravadona Nov 22 '21 at 01:32
  • 2
    Take a look at inotifywait, incron -- and, on a modern platform, systemd path units – Charles Duffy Nov 22 '21 at 01:41
  • @Fravadona, you don't need a `tail`-specific timeout mechanism; the generic ones work as well for `tail` as they work for everything else. See [BashFAQ #68](https://mywiki.wooledge.org/BashFAQ/068). – Charles Duffy Nov 22 '21 at 02:17
  • @CharlesDuffy I was looking at `inotifywait`. You need to watch the parent directory and use the exclude option to match the target file only,which is a little tricky. I didn't figured out how to know if there was something written in the file, I don't think its possible. – Fravadona Nov 22 '21 at 02:30
  • @CharlesDuffy `inotifywait` looks very handy, but it seems unsupported by zsh / bash. – Ugty Nov 22 '21 at 02:53
  • It's not part of any specific shell; it's a command you can install and then use from any shell you want. – Charles Duffy Nov 22 '21 at 03:03
  • @CharlesDuffy I can't make `inotifywait` notify me of the creation of a file named `f` (not any other) inside a watched directory. The ERE seems broken – Fravadona Nov 22 '21 at 03:17
  • @Fravadona, I'd need to see a [mre] to be sure I understand the issue you're describing. File a question on the topic? (It's nearing my bedtime but I'll be back tomorrow). – Charles Duffy Nov 22 '21 at 03:22
  • @CharlesDuffy: got my reply here https://stackoverflow.com/a/38288561/3387716 – Fravadona Nov 22 '21 at 11:08
  • FIFOs are all but useless in Docker, since a container normally runs only one process and doesn't share its filesystem with other containers. Can you use a TCP socket instead; at which point the mechanisms in [Docker Compose wait for container X before starting Y](https://stackoverflow.com/questions/31746182/docker-compose-wait-for-container-x-before-starting-y) will work? – David Maze Nov 22 '21 at 12:07
  • @Fravadona, when you said you were writing an ERE, I took that to mean that you understood what was and was not valid in ERE syntax. – Charles Duffy Nov 22 '21 at 12:37
  • @CharlesDuffy: In fact I was correctly using the ERE syntax. With the `exclude` option of `inotifywait` < 3.20.1 I had to negate the filename and a little mistake slipped in in the huge resulting regexp. see https://stackoverflow.com/a/70070254/3387716 – Fravadona Nov 22 '21 at 18:38

1 Answers1

4

If the file doesn't exists then most commands that try to read its content will fail.

To get over that, you can use an until loop with a sleep:

#!/bin/bash

file=/file/to/check

until [ -s "$file" ]
do
    sleep 1
done

# Now we can really start the operations
# ...

This code will test for the existence and non-emptiness of the file every 1 seconds. When the loop exists, you'll be sure that the file exists and is non-empty.

Here's a way to add a timeout:

#!/bin/bash

file=/file/to/check
timeout=30  # seconds to wait for timeout
SECONDS=0   # initialize the bultin counter 

until [ -s "$file" ] || (( SECONDS >= timeout ))
do
    sleep 1
done

[ -s "$file" ] || exit 1 # timed-out

# start the operations
# ...
Fravadona
  • 13,917
  • 1
  • 23
  • 35
  • `$file=...` is not valid syntax for an assignment in bash – Charles Duffy Nov 22 '21 at 01:42
  • Also, you can use `test -s` and not even need to open the file (`-f` only checks existence and file-ness, `-s` also checks nonemptiness) – Charles Duffy Nov 22 '21 at 01:42
  • Wow, I didn't know about the `-s`, thank you. I'm editing my answer to add a timeout – Fravadona Nov 22 '21 at 01:46
  • Thanks for the answer! But it seems that the timeout only waits for the creation of the target file. If I want to have the timeout to include waiting the content of this file (e.g. if the timeout is 30s, the file was created at 10s, but is written until 40s, the process still exits because it passed the timeout). – Ugty Nov 22 '21 at 02:56
  • Oh, you mean to wait for the full content of the file ? That's quite the hard thing to check, you'll need OS specific APIs, or check the size of the file every few seconds to detect no more changes. What OS docker is running? – Fravadona Nov 22 '21 at 03:04
  • @Ugty, tell an inotify-based mechanism to trigger when the file is closed after being written and then you don't need to know or care about timeouts at all. – Charles Duffy Nov 22 '21 at 03:16
  • @Fravadona the docker is using alpine linux os. – Ugty Nov 22 '21 at 03:32
  • @CharlesDuffy "tell an inotify-based mechanism to trigger when the file is closed after being written and then you don't need to know or care about timeouts at all." I don't truly understand it... Would you mind elaborate on it or open a new answer for it? Thank you so much! – Ugty Nov 22 '21 at 03:33
  • @Fravadona " check the size of the file every few seconds to detect no more changes" not exactly... Once the file is being written a non-empty char, the wait exits. I just want to wait until there's any input of this file. – Ugty Nov 22 '21 at 03:35
  • @Ugty `[ -s "$file" ]` test is true when `"$file"` exists **and** has content, so it should do what you want... But, what is a non-empty char? – Fravadona Nov 22 '21 at 03:41
  • @Fravadona ah i see, that's what I wanted. thanks! – Ugty Nov 22 '21 at 03:43