0

I have been noticing behaviour with end of line '\n' I don't understand and therefore I have come up with the following "test". I am on Mac OS X.

I have a file with:

one two
three four five
six seven

Then I run

x=$(awk '{print}' filename.txt)

and

echo "$x"

prints correctly (i.e. the content of the file as above).

However, when I do

y=$(echo $x | tr "\n" "N")

(I do mean to translate new line '\n' into a capital 'N', just to see what happens), then I get

one two three four five six sevenN

It seems that all occurrences of '\n', but the last, are detected (because the output is on one line) but not replaced with 'N', while the replacement occurs for the last instance.

Is the last character (at the end of the file) different from the other new line characters?

I don't understand what is going on.

EDIT

I can think of one difference between the last line and the others. awk sees one new line character \n at the end of all lines except the last one, and when it prints it adds another one such new line. So what I need is to replace a double new line with N.

On the other hand, why do I not see two Ns?

There is only one '\n' at the end of the last line, and it is replaced with N.

Is this getting closer to an explanation?

Antoine
  • 600
  • 7
  • 19
  • the commands as pasted here work for me on mac. Can you try in a new terminal to make sure there is nothing intefering? – redacted Nov 05 '18 at 13:36
  • 2
    The difference is that you didn't quote the `$x` variable. Write `y=$(echo "$x" | tr "\n" "N")` instead. The `"`s around `$x` are not for decoration. –  Nov 05 '18 at 13:55
  • @mosvy ah yes, the quotes... and yet I had been warned! So now, _with_ the quotes the commands work as expected. I still be happy to understand why it was doing what it did with the missing quotes - but that's not so critical now. – Antoine Nov 05 '18 at 17:50
  • 2
    @Antoine [field splitting](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_05). after which pathname expansion (globbing) will be done too -- eg. if you had '*' instead of 'three' in your file, it will be replaced with the filenames from the current directory. –  Nov 05 '18 at 17:58
  • Does this answer your question? [I just assigned a variable, but echo $variable shows something else](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else) – Charles Duffy Sep 11 '22 at 17:50

2 Answers2

0
x=$(awk '{print}' filename.txt)
y=$(echo $x | tr "\n" "N")

In your example code, ${x} is assigned the contents of filename.txt including newlines (\n).

For the assignment of ${y} you echo variable x. In this case echo will remove \n inside x and append one at the end of the line. That is piped to tr. So it converts the one \n it sees to an N.

If you use echo, the line will be terminated with a \n. Should you use printf, the line may not be terminated with a \n. In that case awk will add an additional \n for the last line. In rare cases, this would have the output not match the input.

Credits

Mosvy was first to notice you needed to quote x, for example echo "${x}" | tr to retain the \n inside x.
RARE Kpop Manifesto noticed the rare case of input not matching output.

James Risner
  • 5,451
  • 11
  • 25
  • 47
0

the trailing % is to indicate it lacks trailing \n :

mawk 'FS = "^$"' RS='\r?\n' ORS='N' 

one twoNthree four fiveNsix sevenN%

skip the RS part if u don't need to account for Windows/DOS sources, and make it ultra succinct :

nawk 'ORS = "N"'

if u want it with trailing \n, and input isn't gigantic :

gawk 'BEGIN { RS="^$"; FS="\r?\n"; OFS="N" } NF=NF'

one twoNthree four fiveNsix sevenN
RARE Kpop Manifesto
  • 2,453
  • 3
  • 11