46

How do get the first column of every line in an input CSV file and output to a new file? I am thinking using awk but not sure how.

kenorb
  • 155,785
  • 88
  • 678
  • 743
Junba Tester
  • 801
  • 2
  • 9
  • 15

6 Answers6

85

Try this:

 awk -F"," '{print $1}' data.txt

It will split each input line in the file data.txt into different fields based on , character (as specified with the -F) and print the first field (column) to stdout.

Levon
  • 138,105
  • 33
  • 200
  • 191
  • 3
    @downvoter .. A downvote ***without*** explanation doesn't help *anyone* (OP, SO or me). This is a functional solution that meets OP's stated requirements. I am happy to correct errors or improve my answer but that requires constructive feedback. – Levon Jul 26 '12 at 11:53
  • 2
    I didn't downvote, but I also won't upvote: It's the use of `awk` where `cut` would do. It smacks of one-size-fits-all-ism; using `perl` or `sed` would be just as bad. Not wrong, just not really right. Now, if you had answered with an `awk` script that handled a csv file like `"last, first",field2,field3` correctly, that would have been more appropriate. – sorpigal Jul 26 '12 at 13:05
  • 3
    @Sorpigal ..and I wouldn't have downvoted *you* if you had used `cut` in place of `awk` :-) .. either tool is fine for this. FWIW, OP mentioned awk in their post, and I upvoted a "competing" `cut` solution (it could have been yours had you posted). It's not a religion, it's a small task that needed to be done, and I picked one of several tools to do it. – Levon Jul 26 '12 at 13:26
  • @Levon May be the down-voter saw your solution as an incomplete one. OP wanted the output to a new file. :P – jaypal singh Jul 26 '12 at 16:20
  • @JaypalSingh Ha ha .. yes, perhaps, but that would be somewhat petty (anyone using a linux system most likely would know how to use io redirection) and could have easily been noted by the downvoter (and then trivially fixed). OP didn't seem troubled by that (nor do all of the answers provide this). Doesn't matter, it solved OP's problem which is main reason for the Q&A. – Levon Jul 26 '12 at 16:25
  • @Levon: I was trying to suggest a motivation for a down vote, that's all. There was no need for me to post anything since the topic had already been covered sufficiently and completely before I saw it. – sorpigal Jul 26 '12 at 18:39
  • I am a total newbie to Shell scripting. Can anyone explain me how to write this when the separation is tab instead of comma? – DarkRose May 05 '16 at 09:03
  • @DarkRose I'm pressed for time right now, so can't test it, but try using `\t` in place of the comma above – Levon May 05 '16 at 13:27
69

Can be done:

$ cut -d, -f1 data.txt
  • This is by far the fastest of all the answers, for a large CSV file. My situation involves a 2GB file containing rows that look like `2021-12-26,472406,616125`. To get the first column, this answer using cut takes 5.1 seconds. Awk (`awk -F, '{print $1}'`) takes 40 seconds. Perl (`perl -F, -lane 'print $F[0]'`) takes 49 seconds. Ripgrep (`rg -o '^[^,]+'`) takes 27 seconds. GNU grep (`grep -o '^[^,]\+'`) takes 177 seconds. – dtolnay Aug 10 '22 at 05:47
12
echo "a,b,c" | cut -d',' -f1 > newFile
Nykakin
  • 8,657
  • 2
  • 29
  • 42
  • 3
    The `'`s around the delimiter are not necessary if the shell can handle it unescaped. –  Jul 26 '12 at 11:54
  • 1
    +1 to counter the down vote. This answer is arguably the most complete and correct! – sorpigal Jul 26 '12 at 18:40
5

Input 

a,12,34
b,23,56

Code 

awk -F "," '{print $1}' Input

Format

awk -F <delimiter> '{print $<column_number>}' Input
Debaditya
  • 2,419
  • 1
  • 27
  • 46
1

This can be achieved using grep:

$ grep -o '^[^,]\+' file.csv
kenorb
  • 155,785
  • 88
  • 678
  • 743
-1

Using Perl:

perl -F, -lane 'print $F[0]' data.txt > data2.txt

These command-line options are used:

  • -n loop around every line of the input file
  • -l removes newlines before processing, and adds them back in afterwards
  • -a autosplit mode – split input lines into the @F array. Defaults to splitting on whitespace.
  • -e execute the perl code
  • -F autosplit modifier, in this case splits on ,

If you want to modify your original file in-place, use the -i option:

perl -i -lane 'print $F[0]' data.txt

If you want to modify your original file in-place and make a backup copy:

perl -i.bak -lane 'print $F[0]' data.txt

If your data is whitespace separated rather than comma-separated:

perl -lane 'print $F[0]' data.txt

Chris Koknat
  • 3,305
  • 2
  • 29
  • 30