7

In bash, this works:

echo -n $'a\nb\nc\n' | while read x; do echo = $x =; done

The while loops through three times

= a =
= b =
= c =

But imagine a text file that doesn't have the conventional trailing newline. I think that read should still work for all three lines, but it doesn't. I just get:

echo -n $'a\nb\nc' | while read x; do echo = $x =; done

= a =
= b =

The help read in bash doesn't really clarify.

Note: I don't need this resolved, and I can see some ways to fix it myself. I am curious, and I am tempted to file a bug report - I generally try myself to respect files that mightn't have the trailing new line. I came across this when using the -d option to read. read -d " " will split on spaces instead of newlines, but it will miss out on the last entry unless it has a trailing space.

(Ubuntu. GNU bash, version 4.1.5(1)-release)

codeforester
  • 39,467
  • 16
  • 112
  • 140
Aaron McDaid
  • 26,501
  • 9
  • 66
  • 88
  • for what it's worth, your 2nd script works as the first with ksh. Good luck. – shellter Dec 22 '11 at 22:06
  • I just checked: `ksh` behaves the same way as `bash` in this case, @shellter. – codeforester Jan 12 '21 at 00:22
  • 1
    @codeforester : I'm sure I tested this before posting, but no longer have access to the UWIN ksh93. Also it is possible that the version I was using is different that the version you are using, although I'm almost sure I saw this very issue come up in the UWIN/ksh email groups (~<10 yrs ago ;-) ). I like your alternate below. Cheers. – shellter Jan 12 '21 at 00:52

2 Answers2

9

If you want the above loop to process the incomplete line, do this:

echo -n $'a\nb\nc' | while read x || [[ $x ]]; do echo = $x =; done

which gives:

= a =
= b =
= c =

When read encounters the incomplete line, it does read that into the variable (x in this case) but returns a non-zero exit code which would end the loop, and || [[ $x ]] takes care of running the loop for the incomplete line as well. When read is called the next time, there is nothing to read and it exits with 1, setting x to an empty string as well, which ensures that we end the loop.

Related

codeforester
  • 39,467
  • 16
  • 112
  • 140
1
$ man bash
   read [-ers] [-a aname] [-d delim] [-i text] [-n nchars] [-N nchars] [-p prompt] [-t timeout] [-u fd] [name ...]
          One line is read from the standard input, ...

I think the key is: How to define "One line".
Does text without a '\n' at the end makes One line?
I guess read don't think so.

kev
  • 155,172
  • 47
  • 273
  • 272
  • 1
    My text editor thinks it's a line, and I think every text editor will. So if the text editor can deal sensibly with files which don't have a newline as their very last character, then I think `read` should too :-) – Aaron McDaid Dec 22 '11 at 12:45
  • `read` is correctly handling it - it reads every valid line. It doesn't read incomplete lines as it has no idea whether they're "lines", an interrupted stream, a broken pipe, etc.... Spec: [a line](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206); Spec: [an incomplete line](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_195); `read` doesn't have the luxury of being able to assume that everything it operates on is a complete file intended for humans. Standards aren't based on what (often buggy) text-editors think. – Mark K Cowan Feb 13 '23 at 12:14