bash file reading by line; simultaneously allow interactive stdin?

Question

I think I'm pushing the limits of bash here but I really want to get this done without having to re-write the entire script.

I have something dynamic that is a list of pairs of files which I want to perform operations on. I also want to prompt before I continue with any of the tasks.

It looks like this:

diff -rq $dir1 $dir2 | \
    sed -ne 's/^Files \(.*\) and \(.*\) differ$/\1 \2/p' | \
    while read differingfilepair; do
        ...
        printf "Continue? (Y/n)"
        read -n1 cont
    done

As you can see here the while read line block appears to function as some form of a subshell which receives the content of the data over STDIN.

The $cont variable basically just slurps the first char of each line of that data (the first char in the path of the first file in the pair of files reported by diff to be differing), it's not attached to the terminal.

Is there a way to do what I'm trying to do here? I guess a workaround is to use a temporary file but is there another way?

Edit: Using a temporary file later loaded into the while read block like this: while read l; do ...; done < .tmp_file still takes over stdin! (though I think this answer will help)

Yes that seems to be part of the equation for a file descriptor redirection approach to this issue... it's a bit beyond me at this point, though. I ended up leveraging the structuredness of my data and I got by by just flattening my list and making the inner loop track the state. Used a `for file in \`cat file\`` — Steven Lu, Dec 18 '13 at 23:20

score 1 · Answer 1 · edited May 23 '17 at 12:03

1

I think this just might do the trick:

read -n1 cont </dev/tty

Possible duplicate of Read input in bash inside a while loop

edited May 23 '17 at 12:03

Community

1
1

answered Dec 18 '13 at 23:21

Donovan

15,917
4
22
34

Reinstate Monica Please · Accepted Answer · 2013-12-19T00:11:46.290

1

In bash file descriptor 1 is stdin, 2 is stderr, and after 9 can be used by internal shell processes. (from bash man page) (see http://mywiki.wooledge.org/BashFAQ/089 for examples)

Redirections using file descriptors greater than 9 should be used  with
care,  as they may conflict with file descriptors the shell uses inter‐
nally.

So you can read your files into 3-9 e.g. (replace echo with however you're getting files)

for file in $(echo "file1.txt file2.txt"); do
  while IFS='' read differingfilepair <&3; do
    ...
    printf "Continue? (Y/n)"
    read -n1 cont
  done 3< "$file" #Or some command that gets file names
done

edited Dec 19 '13 at 00:11

answered Dec 18 '13 at 23:22

Reinstate Monica Please

11,123
3
27
48

Excellent. Could you also note how to use a different file descriptor to pipe something into the `while`? – Steven Lu Dec 18 '13 at 23:24
1

Perfect answer. Only critique is that TLDP probably isn't the best reference to point folks at, as it often tends to be outdated or inaccurate. http://mywiki.wooledge.org/BashFAQ/089 is another reference directly on-point. – Charles Duffy Dec 18 '13 at 23:30
@Steven Lu Do you have to pipe it? Getting a bad file descriptor warning when I tried it a second ago. I'm sure there is some magic way to get it, but may just be easier to sequentially redirect them in. – Reinstate Monica Please Dec 18 '13 at 23:53
@Charles Duffy Thanks, added to post for reference – Reinstate Monica Please Dec 19 '13 at 00:07
re: recent edits: `for file in $(echo ...)` is actually bad practice; see entry #1 in http://mywiki.wooledge.org/BashPitfalls, and how it applies to find &c. also. – Charles Duffy Dec 19 '13 at 04:36
Hmm. I guess sequential redirection is how e.g. implementing this (quite straightforwardly I might add) in Python would do it right? How exactly would a scripting language intepreter relinquish stdin to read a file and then get it back? – Steven Lu Dec 19 '13 at 04:37
...also, bash 4.2 supports automatic FD allocation, so you can safely get more than 10 FDs by asking the shell to give you a free one: `exec {new_fd} – Charles Duffy Dec 19 '13 at 04:37
@StevenLu well, idiomatic use of files in Python isn't using `sys.stdin` at all, but is using a different FD number for its reads. You can do that in bash, too: `read -u "$new_fd"` will use the FD number in `new_fd`, rather than relying on FD 0 to be redirected. – Charles Duffy Dec 19 '13 at 04:38

bash file reading by line; simultaneously allow interactive stdin?

2 Answers2