1

I am doing find-replace operations using "sed" in Linux. I have a XML file in Linux named config.xml. The file contains data as followed-

<CATALOG>
 <SERVER>
  <URL value="http://ip-172-44-0-92.compute.internal:440/" />
 </SERVER>
</CATALOG>

I want to find a line in the config.xml file that contains <URL value= and replaces the entire line with <URL value="http://ip-181-40-10-72.compute.internal:440/" />

I tried it by executing the command-

sed -i '/<URL value=/c\<URL value=\"http://ip-181-40-10-72.compute.internal:440/\" />' config.xml

The command executes correctly and does find replace operation but, when I open the file using vi config.xml I see ^M character at the end of each lines that were not replaced. Why did this happened and, how to fix it?

EDIT-

By referring to @Atalajaka's answer... My original file contains CRLF line endings at the end of each line. And, sed replaces the line with LF line ending. As a result, all other unreplaced lines will still have CRLF ending and 'vi' editor will now show ^M at the end of these lines.

So the solution is to replace CRLF endings with LF endings by running the command-

sed -i $'s/\r$//' config.xml
YASH JADHAV
  • 143
  • 2
  • 11
  • 1
    Does every line of the file show `^M` or only the lines that were not replaced? Maybe your file contains DOS line endings, i.e. CR and LF, and your replacement uses UNIX line ending, i.e. LF only. This might change how the file is displayed in `vi`. Please [edit] your question and show the output of `od -c config.xml` before and after your modification. – Bodo Apr 13 '22 at 10:36
  • @Bodo Only the lines that were not replaced shows `^M`. – YASH JADHAV Apr 13 '22 at 10:43
  • See also [Why does my tool output overwrite itself and how do I fix it?](https://stackoverflow.com/questions/45772525/why-does-my-tool-output-overwrite-itself-and-how-do-i-fix-it) – Sundeep Apr 13 '22 at 11:10
  • 1
    Please don't put an answer into your question. If you want to answer your question yourself you should write an answer. (Yes, you can answer your own question.) – Bodo Apr 14 '22 at 10:30

1 Answers1

1

The issue could rely on the original XML file, in case it has CRLF endings for each line. If that is the case, vim will recognize and hide them, so that they remain unimportant to you. Assuming this, all new lines added with vim will contain those same CRLF line-endings.

The sed command uses LF line-endings when adding any new lines, so when that happens, vim sees the two different line-endings, and will assume the LF line-endings as the regular ones. This means that all CR line-endings will be displayed as ^M.

If you have access to the original XML file before being edited, you can open it in vim and check if you see [dos] at the footer, right next to the file name, something as:

$ vim original_xml.xml

...

"original_xml" [dos] ...

Source.

Atalajaka
  • 125
  • 1
  • 2
  • 14
  • 1
    Yes I do see [dos] in the footer. You are right the original file contains CRLF endings at each line. I need to replace CRLF endings with LF endings. Thanks! – YASH JADHAV Apr 13 '22 at 11:16
  • 1
    You are very welcome! If this answer solved your issue, please consider accepting it. That way any future visitors to this question will know at a glance that this is a valid solution to the problem. – Atalajaka Apr 13 '22 at 13:42
  • 1
    This answer does not yet show how to fix the problem. One option would be to use `dos2unix config.xml` before the `sed` command and, if necessary `unix2dos config.xml` afterwards. It would also be possible to change the `sed` command to keep the CR character if present. – Bodo Apr 13 '22 at 14:46
  • @Bodo Just execute the command `sed -i $'s/\r$//' config.xml`. It will replace CRLF line endings with LF line endings. – YASH JADHAV Apr 14 '22 at 07:59
  • @YASHJADHAV I don't need an explanation how `dos2unix` can be replaced with a (longer) `sed` command. My previous comment was a hint for the author of this answer how it can be improved to actually answer the second part of your question *"Why did this happened and, how to fix it?"* – Bodo Apr 14 '22 at 10:31