-6

I want to extract those lines a csv file which match a Pattern and then append the same Pattern to the end of each extracted line as a newly added column of the csv file.

file.csv

file.csv

/var/log/0,33,New file,0
/var/log/0,34,Size increased,2345
/abc/Repli,11,New file,0
/abc/Repli,87,Size Increase,11

In above file file.csv, I executed

sed -n -i"" '/Repli/ s/$/,Repli/p' file.csv

This deletes remaining lines, which I do not want.

Tany3108
  • 1
  • 2
  • You might want to add some more details to this question. Maybe post some code of what you've tried. – Ron Aug 12 '14 at 04:50
  • Do you want to append the pattern (which could be a regex?), or the value that the pattern matches? – mhawke Aug 12 '14 at 05:09
  • I have tried. But every time it adds a control character ^M and in excel it shows in next row instead of next column – Tany3108 Aug 12 '14 at 05:34
  • @mhawke I want to append based on matching with the pattern. For ex. Match /var/log/0/, if found, append BE in last column of matched lines – Tany3108 Aug 12 '14 at 05:36
  • Are there other cases, e.g. if match /var/log/1 append XYZ ? – mhawke Aug 12 '14 at 06:19
  • Right. This is exactly the case. sed -i"" 's/\r//; /pattern/ s/$/,pattern/' file.csv is working perfect for single word patterns, but for paths it is giving me sed syntax error – Tany3108 Aug 12 '14 at 06:23

3 Answers3

0

Extracting only the lines that match a pattern and modifying them

To select only lines containing pattern and then add pattern as a new column at the end of the line:

awk '/pattern/ {print $0 ",pattern"}'  file.csv >tmp$$ && mv tmp$$ file.csv

Or, 

sed -b -n -i"" '/pattern/ s/$/,pattern/p' file.csv

Keeping all lines but modifying those that match a pattern

awk '/pattern/ {$0=$0 ",pattern"} 1'

Or, 

sed -b -i"" '/pattern/ s/$/,pattern/' file.csv

Remove Windows line endings while keeping all lines and modifying those that match a pattern

sed -i"" 's/\r//; /pattern/ s/$/,pattern/' file.csv

Remove Windows line endings while keeping all lines and modifying those that match a pattern containing slashes

Suppose that the pattern contains slashes like /var/log/abc/file/0/. Then:

sed -i"" 's/\r//; \|pattern| s|$|,pattern|' file.csv

For example:

sed -i"" 's/\r//; \|/var/log/abc/file/0/| s|$|,/var/log/abc/file/0/|' file.csv
John1024
  • 109,961
  • 14
  • 137
  • 171
  • I Tried awk '/pattern/ {print $0 ",pattern"}' file.csv. But it gives me the pattern appended at the beginning of the line and not the end. Also, I require to have changes in the file in place. Using awk is just giving me changes output. – Tany3108 Aug 12 '14 at 04:58
  • See updated answer for changing the file in place. `print $0 ",pattern"` will put the pattern at the end of the line. If something else happened when you tried it, please document exactly what you did with sample input, the code, and sample output. – John1024 Aug 12 '14 at 05:07
  • sed -n -i"" '/pattern/ s/$/,pattern/p' file.csv adds a control character before adding the pattern and thus when I open this csv in excel, the last column actually gets displayed in next row. But I want to display it in same row. – Tany3108 Aug 12 '14 at 05:12
  • That should not happen. Please document exactly what you did with sample input, the code, and the output in which the control character appeared. – John1024 Aug 12 '14 at 05:18
  • sed -n -i"" '/pattern/ s/$/,pattern/p' file.csv was run on the file due to which in file.csv only lines containing pattern remained and other lines deleted. Also ^M was added before pattern in each matched line as '^M, pattern' – Tany3108 Aug 12 '14 at 05:33
  • OK. That would appear to mean that you are on Windows and your `sed` [has a bug in it](http://stackoverflow.com/questions/4652652/sed-preserve-line-endings). As a work around, try: `sed -b -n -i"" '/pattern/ s/$/,pattern/p' file.csv` – John1024 Aug 12 '14 at 05:42
  • I am working on bash shell on RHEL5.1. Running sed -b did not work on it giving error. Also, I want only matched lines to append in the file. The rest of the lines should remain as it is. In current usage of sed -n -i"" '/pattern/ s/$/,pattern/p' file.csv, remaining lines not matching the pattern get deleted. – Tany3108 Aug 12 '14 at 05:52
  • The ^M characters indicate Windows line endings. If they are appearing on RHEL, that indicates a problem with `file.csv`. Run it through a dos2unix converter. – John1024 Aug 12 '14 at 05:59
  • sed -i"" 's/\r//; /pattern/ s/$/,pattern/' file.csv seems to have worked for me. Thanks a lot. – Tany3108 Aug 12 '14 at 06:09
  • If you could please answer a small query as well that if pattern is complex like "pattern=/var/log/abc/file/0/", then how can I match it using sed. I tried simply sed -n -i"" '//var/log/abc/file/0/ s/$/,pattern/p' file.csv gave me error – Tany3108 Aug 12 '14 at 06:20
0

I found a solution to match paths using sed. I did it through escap character and it worked.

Pattern="\/var\/log\/Model\/1\/" Module=BE sudo sed -i"" "s/\r//; /$Pattern/ s/$/,$Module/" resultFile.csv

Worked Fine!!

Tany3108
  • 1
  • 2
0

Below snippet appends the pattern in case if a match is found. If no match found just prints the line,

awk '{if($0 ~ /pattern/) print $0",pattern"; else print $0;}' file.csv
Sadhun
  • 264
  • 5
  • 14