0

I am having a weird behavior with bash and sed (on a Debian9 machine), it may be something related to the file (file1.CSV) I'm using but I cannot figure out what.

Here we go, if I try to replace the endline with something through sed it mismatch the linestart (that is a "):

head app/datasets/liste_random/file0.csv  | sed 's/$/@/'

@id","stratum","block.id","block.size","treatment"
@PC_LR01","Piacenza_LR","PC_LR1",4,"COLC"
@PC_LR02","Piacenza_LR","PC_LR1",4,"COLC"
@PC_LR03","Piacenza_LR","PC_LR1",4,"SOC"
@PC_LR04","Piacenza_LR","PC_LR1",4,"SOC"
@PC_LR05","Piacenza_LR","PC_LR2",4,"SOC"
@PC_LR06","Piacenza_LR","PC_LR2",4,"COLC"
@PC_LR07","Piacenza_LR","PC_LR2",4,"COLC"
@PC_LR08","Piacenza_LR","PC_LR2",4,"SOC"
@PC_LR09","Piacenza_LR","PC_LR3",2,"SOC"

I looked for possible reasons but I could find nothing interesting, sure it is something stupid but I could not figure out what was the problem.

If I echo a line it works properly, so think it is the file

echo '"id","stratum","block.id","block.size","treatment"
"PC_LR01","Piacenza_LR","PC_LR1",4,"COLC"'  | sed 's/$/@/g'

"id","stratum","block.id","block.size","treatment"@
"PC_LR01","Piacenza_LR","PC_LR1",4,"COLC"@

Thank you so much in advance for any help!

cccnrc
  • 1,195
  • 11
  • 27
  • 3
    Your files have DOS newlines -- that is to say, two-character `` ("carriage return" / "linefeed") sequences, as opposed to UNIX newlines, which have only a ``. Thus, `sed` is adding content after the `` and before the ``, so it's printed at the beginning of each line (after the carriage has been returned -- the cursor moved to the left -- but before the next line is fed -- the cursor moved down). – Charles Duffy Apr 29 '20 at 00:25
  • 1
    Run `dos2unix` on your file to remove the carriage returns. – Quasímodo Apr 29 '20 at 00:27
  • 1
    Yeah, crlf, is the culprit, or maybe `tr -d "\r" < file1.CSV | sed 's/$/@/'` – Jetchisel Apr 29 '20 at 00:27
  • 3
    (as context for what Jetchisel is suggesting above, `\r` is another way to write the `` character, just as `\n` is the equivalent way to express an ``; thus, one could also say that UNIX newlines are just `\n`, whereas DOS/Windows ones are `\r\n`). – Charles Duffy Apr 29 '20 at 00:30

0 Answers0