Closest I could find to this is Is it possible to mimic process substitution on msys /mingw (with bash 3.x) - but I cannot really get it to work in my case; so let me demonstrate.
I'm using:
$ bash --version
GNU bash, version 5.1.8(1)-release (x86_64-pc-msys)
I usually have a specific use for process substitution and diff
(as in the linked question) - usually I want to call head
or tail
to compare parts of files. As a starting point we might have:
cat > /tmp/tst <<EOF
line 1
line 2
line 3
EOF
So, I might want to call something like this - which I think is great, because it is a (readable for me) one-liner, which allows pre-processing files for diff
:
$ diff <(head -n 3 /tmp/tst) <(head -n 2 /tmp/tst)
3d2
< line 3
This works here - however, for network storage with large files it might fail ... additionally, note that process substitution usually replaces the <(...)
with a filename:
$ echo <(head -n 3 /tmp/tst)
/dev/fd/63
... however, some programs might receive the "Windows" path, as in via cygpath -w
:
$ echo $(cygpath -w <(head -n 3 /tmp/tst))
/proc/694/fd/pipe:[249108113544]
... like for instance meld
(MINGW64):
meld <(head -n 3 /tmp/tst) <(head -n 2 /tmp/tst)
The above command starts meld
- and we would have expected /dev/fd/*
filenames in the call to meld
- however, when meld
opens, there are the following messages in the two panes:
There was a problem opening the file “\proc\701\fd\63”. | There was a problem opening the file “\proc\701\fd\63”.
Error opening file \proc\701\fd\62: No such file or directory | Error opening file \proc\701\fd\62: No such file or directory
... so, meld
ultimately received the /proc/*
filenames, as if passed via cygpath -w
- and failed with those filenames.
So, I thought - how about using file descriptors directly? As an example:
## note, fd 3 is typically used in MSYS2:
$ ls /dev/fd/
3@ 0@ 1@ 2@
## assign fd 10 (open for reading) to the process substitution of `head`:
# ( note that `exec 10><(head -n 2 /tmp/tst)` causes "bash: /dev/fd/63: Permission denied" )
# https://stackoverflow.com/questions/7082001/how-do-file-descriptors-work
# https://unix.stackexchange.com/questions/208758/how-to-set-pipe-of-process-after-process-substitution-in-bash-so-that-process-co
$ exec 10< <(head -n 2 /tmp/tst)
## check descriptors
$ ls /dev/fd/
3@ 0@ 1@ 10@ 2@
## check once more /dev/fd/ for fd 10:
$ ls /dev/fd/10
/dev/fd/10@
$ ls -la /dev/fd/10
lrwxrwxrwx 1 user None 0 Jan 14 13:59 /dev/fd/10 -> 'pipe:[515396091548]'|
## cat/output fd 10 via /dev/fd/ filename does NOT work!:
$ cat /dev/fd/10
cat: /dev/fd/10: No such file or directory
## cat/output fd 10 via redirection - replacing stdin of cat - works:
$ cat <&10
line 1
line 2
## cat second time - nothing is printed, since process already finished
$ cat <&10
$
## close fd 10 (close for reading, since it was open for reading)
# http://mywiki.wooledge.org/BashFAQ/085
# https://unix.stackexchange.com/questions/13724/file-descriptors-shell-scripting
$ exec 10<&-
## check descriptors again:
$ ls /dev/fd/
3@ 0@ 1@ 2@
The above snippet shows the core of the problem - how come, that /dev/fd/10
exists when used with ls -la
- but when used with cat
, it seemingly does not exist ("No such file or directory")?
Otherwise, I would have hoped something like this - using /dev/fd/* filenames, when redirecting process output to file descriptor - would have worked for a one-liner (where I can ultimately preprocess the input files for diff
):
$ exec 10< <(head -n 3 /tmp/tst) && exec 11< <(head -n 2 /tmp/tst) && ls /dev/fd/ && ls -la /dev/fd/10 /dev/fd/11 && exe
c 10<&- && exec 11<&- && ls /dev/fd/
3@ 0@ 1@ 10@ 11@ 2@
lrwxrwxrwx 1 user None 0 Jan 14 14:02 /dev/fd/10 -> 'pipe:[528280993436]'|
lrwxrwxrwx 1 user None 0 Jan 14 14:02 /dev/fd/11 -> 'pipe:[532575960732]'|
3@ 0@ 1@ 2@
... however, as seen with the /dev/fd/ filename problem above, this invocation fails with diff
(and fails also if you replace diff
with cat
below):
$ exec 10< <(head -n 3 /tmp/tst) && exec 11< <(head -n 2 /tmp/tst) && ls /dev/fd/ && diff /dev/fd/10 /dev/fd/11 && exec
10<&- && exec 11<&- && ls /dev/fd/
3@ 0@ 1@ 10@ 11@ 2@
diff: /dev/fd/10: No such file or directory
diff: /dev/fd/11: No such file or directory
So, ultimately, my question is - is there a way to use a file descriptor instead of a filename in a bash call of a command? (because if so, I could use file descriptors instead of process substitution in bash
to run one-liner operations on truncated files, where process substitution - while otherwise working - fails on some programs on MSYS2).