1

Hello everyone I'm a beginner in shell coding. In daily basis I need to convert a file's data to another format, I usually do it manually with Text Editor. But I often do mistakes. So I decided to code an easy script who can do the work for me. The file's content like this

/release201209
a1,a2,"a3",a4,a5
b1,b2,"b3",b4,b5
c1,c2,"c3",c4,c5

to this:

a2>a3
b2>b3
c2>c3

The script should ignore the first line and print the second and third values separated by '>' I'm half way there, and here is my code

#!/bin/bash

cat $1 | sed '1d' | cut -d, -f2-3 | tr -d '"' > $2

It was working well until I found out that it is not working for a type of data containing comma in a3 like this one: data,VERSION,"FUNDS.TRANSFER,ASS.VERS.TIERS.BOP",, Which returns VERSION>FUNDS.TRANSFER instead of VERSION>FUNDS.TRANSFER,ASS.VERS.TIERS.BOP

Can you help me out updating it please ? Thanks

woodyboow
  • 41
  • 2
  • Based on [this answer](https://stackoverflow.com/a/29650812/12957340) `awk -vFPAT='([^,]*)|("[^"]+")' '{print $2 ">" $3}' file` will do it – jared_mamrot Jan 17 '21 at 22:50
  • 1
    Does this answer your question? [Escaping separator within double quotes, in awk](https://stackoverflow.com/questions/7804673/escaping-separator-within-double-quotes-in-awk) – jared_mamrot Jan 17 '21 at 22:51
  • It returns awk: invalid -v option – woodyboow Jan 17 '21 at 23:02
  • Try GNU awk: https://www.gnu.org/software/gawk/manual/html_node/Installation.html – jared_mamrot Jan 17 '21 at 23:23
  • 1
    As an aside, that's a [useless `cat`](https://stackoverflow.com/questions/11710552/useless-use-of-cat), and you should [quote your variables](https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable). – tripleee Jan 18 '21 at 05:26

3 Answers3

2

Consider using a proper CSV parsing tool like csvtool to extract the relevant columns (its much easier & more reliable than rolling out your own parsing). Then, use tr/sed to do the necessary transformations:

sed '1d' file.txt | csvtool -t ',' col 2,3 - | tr -d '"' | sed 's/,/>/'

Steps:

  • Remove the header line using sed
  • Use csvtool to extract the 2nd and 3rd columns
  • Use tr to remove the double quotes
  • Use sed to map the first , to a > (you can't use tr for this since that does a global translation)

You can install csvtool with your package manager, e.g. on a Debian-based system sudo apt-get install csvtool. Replace apt-get with your package manager on other systems e.g. yum, brew, ...

costaparas
  • 5,047
  • 11
  • 16
  • 26
1

Ruby has a CSV module included:

ruby -rcsv -e '
  CSV.read(ARGV.shift).map {|row| 
    printf "%s>%s\n", row[1], row[2]
  }
' file | sed 1d
glenn jackman
  • 238,783
  • 38
  • 220
  • 352
0

enter image description here

hello sir I'm also learning a shell script, what do you want like that ??

this is my code

function klir(){
line2=$(cut -d\, -f1 $1 > $1.temp.filterline2)
line3=$(cut -d\" -f2 $1 > $1.temp.filterline3)
paste $1.temp.filterline2 $1.temp.filterline3 | sed "s/\t/>/g ; 1d"
rm $1.temp.filterline2 $1.temp.filterline3 2>/dev/null 
}
klir $1
Gua nub
  • 1
  • 1
  • 1
  • hello, thank you for your answer I want something like that VERSION>FUNDS.TRANSFER,ASS.VERS.TIERS.BOP secondValue>thirdValue – woodyboow Jan 17 '21 at 23:07
  • 1
    Plase [don't post images of text.](https://meta.stackoverflow.com/questions/303812/discourage-screenshots-of-code-and-or-errors) – tripleee Jan 18 '21 at 05:27