Reading a specific column in a CSV file with bash

Question

I'm really inexperienced with bash/shell scripts. I have a large volume of files that I need to SCP across servers. I have a list of the files I need to move in a CSV file with three columns. The second column is called file_name, and contains the path of the file.

I need to read every single line of the file, but only process the second column. I found something like this on the internet:

#!/bin/bash

csv_file=$1

while IFS=',' read -r file_name; do
    echo -e "File Name\t: $file_name"
done < $csv_file
IFS=$' \t\n'

But it just seems to output all 3 columns of the file. Any guidance would be greatly appreciated, I really don't know much about bash.

Thanks!

Why the downvote and close vote for "insufficient information provided"? Seems pretty straightforward to me. — Blue Ice, Dec 03 '14 at 16:36
possible duplicate of [bash, extract one column of a csv file](http://stackoverflow.com/questions/19602181/bash-extract-one-column-of-a-csv-file) — tripleee, Dec 03 '14 at 17:02

score 7 · Accepted Answer · answered Dec 03 '14 at 16:37

7

You need to use a variable for each column of the file being read. The last variable will get all the remaining columns. So it should be:

while IFS=, read -r col1 file_name col3

answered Dec 03 '14 at 16:37

Barmar

741,623
53
500
612

score 2 · Answer 2 · answered Dec 03 '14 at 16:47

If the output being in another singe file is acceptable, his sounds like something well suited for awk, which is a program built for dealing with delimited input line by line.

For example, the below may help with your problem, if I'm understanding it correctly:

awk -F',' '{print "File Name:\t"$2}' inputfile > outputfile

The -F',' tells awk the field separator in the file is a comma.

print is awk's command to print output, in this case the string "File Name:\t" followed by the second field in the file, which is denoted in awk by $i where i is the field number (NOT index, $0 for awk is the entire line) you're interested in.

awk is a very powerful tool and this is just a very simple example of the complex processing it can do - you can read more about it's features and capabilities here: http://www.staff.science.uu.nl/~oostr102/docs/nawk/nawk_toc.html

Reading a specific column in a CSV file with bash

2 Answers2