AWK is inserting in the beginning instead of appending, only with specific input

Question

I wrote a simple awk one liner to append a string before and after each each line of a given file:

awk '{print "--> " $0 " <--"}' filename

Now I have a test file containing these lines:

test
test2
test3

The result is as expected:

--> test <--
--> test2 <--
--> test3 <--

However, these are my real lines I need to process:

"\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b"
"\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c"
"\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c"

Processing those with the very same one liner will not work, it will instead only insert the rightmost arrow in front of the line.

 <--"\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b"
 <--"\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c"
 <--"\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c"

Why is this happening and how can I work around it?

I concur with @Fravadona. Moreover you may not be seeing the whole output due to overlap on the console. — Kaz, Dec 02 '22 at 23:43
Your awk procedure works fine on the same data on my system (GNU Awk 5.1.0, API: 3.0, on raspberry pi 400). — Dave Pritlove, Dec 02 '22 at 23:44

score 0 · Accepted Answer · answered Dec 02 '22 at 23:39

0

Looks like your input file contains dos/windows line endings (\r\n); try ...

head -2 filename | od -c

... and you should see each line ending with \r \n.

Run dos2unix filename to permanently remove the \r characters from the file.

Alternatively you can have awk strip the \r, eg:

awk '{sub(/\r/,""); print "--> " $0 " <--"}' filename

answered Dec 02 '22 at 23:39

markp-fuso

28,790
4
16
36

Nit-pick - `\r` can occur and isn't "special" in a text file unless it's at the end of a line so `sub(/\r/,"")` should really be `sub(/\r$/,"")` to avoid removing the first `\r` in the line and leaving the trailing one. – Ed Morton Dec 03 '22 at 17:52

AWK is inserting in the beginning instead of appending, only with specific input

1 Answers1