Why does rewriting a file with "envsubst file" leave it empty?

Question

I need to put some environment variables values to file.

cat file
# $VAR

echo $VAR
# text

When I do envsubst '$VAR $VAR' < file > file file becomes empty. To resolve this issue I use envsubst '$VAR $VAR' < file | tee file 2>&1 >/dev/null. But sometimes it doesn't work, I mean file becomes empty. The first question is why sometimes file becomes empty and the second is what is the TRUE Unix way to put env vars to file in Linux and *BSD and MacOS ?

`cmd file > file` is 99.9999% always the wrong approach in `*nix` (unix&Linux). Get used to using @anubhava 's pattern for any simple file changes you need to do in `*nix`. Good luck. — shellter, Nov 02 '17 at 17:52
Not *remotely* specific to `envsubst`. You can have this problem with *literally any* command when you're both reading from and writing to the same file via redirection. — Charles Duffy, Nov 02 '17 at 20:22

score 37 · Accepted Answer · answered Nov 02 '17 at 20:19

The shell always processes the output redirection, which truncates the file, before starting the command.

With the pipeline envsubst ... < file | tee file, the two commands envsubst and tee run asynchronously, which means the OS is free to start one or the other command first. You are basically tossing the dice that envsubst runs and reads file before tee runs and truncates the file.

The only correct solution is to write to a temporary file, then replace the original with the temporary file after you have confirmed that the command was successful.

Why does rewriting a file with "envsubst file" leave it empty?

1 Answers1

Linked