How can I use ":" as an AWK field separator?

Question

Given the following command,

echo "1: " | awk '/1/ -F ":" {print $1}'

why does AWK output:

1:

?

`awk '/1/ -F ":" {print $1}'` will **ALWAYS** print out `$1` or an empty row if input row is also empty, ***regardless*** of whether `"1"` actually exists in that row or not — RARE Kpop Manifesto, Sep 10 '22 at 17:54

score 562 · Accepted Answer · edited Oct 14 '22 at 16:39

562

-F is a command line argument, not AWK syntax. Try:

echo '1: ' | awk -F  ':' '/1/ {print $1}'

edited Oct 14 '22 at 16:39

CervEd

3,306
28
25

answered Apr 09 '10 at 17:33

Jürgen Hötzel

18,997
3
42
58

53

Ignorant's question here: the /1/ part is to tell awk to only process rows (or records to be more precise) that contain the number 1 right? – rantsh Mar 21 '13 at 19:13
13

@rantsh Awk syntax looks like `(pattern){action}`. If `pattern` (mostly a conditional statement) is _true_, `action` is executed. If `pattern` is not available, `true` is implied. Here the `pattern` is `/1/` which states _is regex `1` matched in the current record `$0`_ – kvantour Jan 10 '20 at 09:43
1

As a sidenote, if you separator is comma, you'd want to add -v OFS="," parameter to awk, to keep it in the output – Altair7852 Mar 02 '21 at 15:11
And how you can indicate this in a .awk script? – Joan Serrano May 23 '21 at 21:27
@JoanSerrano Look at the answer by Dennis below that does exactly that – vmallet May 26 '21 at 19:26

score 80 · Answer 2 · answered Apr 09 '10 at 17:39

If you want to do it programatically, you can use the FS variable:

echo "1: " | awk 'BEGIN { FS=":" } /1/ { print $1 }'

Note that if you change it in the main loop rather than the BEGIN loop, it takes affect for the next line read in, since the current line has already been split.

fedorqui · Answer 3 · 2020-08-27T13:29:18.820

69

You have multiple ways to set : as the separator:

awk -F: '{print $1}'

awk -v FS=: '{print $1}'

awk '{print $1}' FS=:

awk 'BEGIN{FS=":"} {print $1}'

All of them are equivalent and will return 1 given a sample input "1:2:3":

$ awk -F: '{print $1}' <<< "1:2:3"
1
$ awk -v FS=: '{print $1}' <<< "1:2:3"
1
$ awk '{print $1}' FS=: <<< "1:2:3"
1
$ awk 'BEGIN{FS=":"} {print $1}' <<< "1:2:3"
1

edited Aug 27 '20 at 13:29

answered Mar 24 '16 at 23:34

fedorqui

275,237
103
548
598

which is the preferred way? i assume the final example with the `BEGIN` statement would be the most correct (being consistent with the overall `awk` syntax). – Jun 01 '18 at 11:46
1

@randomware all of them are fine. I tend to use `BEGIN` if I use a file to store the whole thing, while `-F` comes in handy with one-liners. – fedorqui Jun 01 '18 at 12:16
1

It must be said that there are subtle differences between the third case and all others. Example : `awk 'BEGIN{print split("foo:bar",a)}' FS=":" file` and `awk 'BEGIN{FS=":"; print split("foo:bar",a)}' file` – kvantour Dec 06 '18 at 19:20
@kvantour good point. I just asked about it in [Why is field separator taken into account differently if set before or after the expression?](https://stackoverflow.com/q/57586974/1983854). – fedorqui Aug 21 '19 at 07:59
1

Thank you! I learn best from clear examples. – Merlin Feb 03 '22 at 18:28
@kvantour : while your statements are very much true, personally, i always explicitly set the splitting delimiter in `split( )`, especially within functions, instead of having it being at the mercy of whatever `FS` is at that very moment (unless i'm splitting an empty string to clear an array, in which case, it doesn't matter at all) – RARE Kpop Manifesto Sep 10 '22 at 17:36

score 12 · Answer 4 · edited Aug 27 '20 at 13:20

12

You can also use a regular expression as a field separator. The following will print "bar" by using a regular expression to set the number "10" as a separator.

echo "foo 10 bar" | awk -F'[0-9][0-9]' '{print $2}'

edited Aug 27 '20 at 13:20

Peter Mortensen

30,738
21
105
131

answered Aug 05 '16 at 20:15

Zlemini

4,827
2
21
23

score 11 · Answer 5 · answered Apr 09 '10 at 17:33

11

-F is an argument to awk itself:

$echo "1: " | awk -F":" '/1/ {print $1}'
1

answered Apr 09 '10 at 17:33

danben

80,905
18
123
145

4

No need to quote colon. – ceving Apr 10 '17 at 10:35

score 5 · Answer 6 · edited Aug 27 '20 at 13:21

5

Or you can use:

echo "1: " | awk  '/1/{print $1-":"}'

This is a really funny equation.

edited Aug 27 '20 at 13:21

Peter Mortensen

30,738
21
105
131

answered Oct 04 '16 at 14:05

Vonton

2,872
4
20
27

1

what does `/1/` mean? – Jun 01 '18 at 11:51
Find a pattern. In this case "1" – José Dias Nov 26 '19 at 09:59
Why is it a really funny equation? – Peter Mortensen Aug 27 '20 at 13:21
i think this trick only works cuz the value before ":" is numeric. echo "ab1: " | awk '/1/{print $1-":"}' fails, printing a "0" – RARE Kpop Manifesto Jun 20 '21 at 08:51

score 4 · Answer 7 · edited Aug 27 '20 at 13:13

4

There isn't any need to write this much. Just put your desired field separator with the -F option in the AWK command and the column number you want to print segregated as per your mentioned field separator.

echo "1: " | awk -F: '{print $1}'
1

echo "1#2" | awk -F# '{print $1}'
1

edited Aug 27 '20 at 13:13

Peter Mortensen

30,738
21
105
131

answered May 02 '15 at 11:33

Bhavuk Taneja

49
1
1

score 4 · Answer 8 · edited Aug 27 '20 at 13:17

AWK works as a text interpreter that goes linewise for the whole document and that goes fieldwise for each line. Thus $1, $2...$n are references to the fields of each line ($1 is the first field, $2 is the second field, and so on...).

You can define a field separator by using the "-F" switch under the command line or within two brackets with "FS=...".

Now consider the answer of Jürgen:

echo "1: " | awk -F  ":" '/1/ {print $1}'

Above the field, boundaries are set by ":" so we have two fields $1 which is "1" and $2 which is the empty space. After comes the regular expression "/1/" that instructs the filter to output the first field only when the interpreter stumbles upon a line containing such an expression (I mean 1).

The output of the "echo" command is one line that contains "1", so the filter will work...

When dealing with the following example:

echo "1: " | awk '/1/ -F ":" {print $1}'

The syntax is messy and the interpreter chose to ignore the part F ":" and switches to the default field splitter which is the empty space, thus outputting "1:" as the first field and there will be not a second field!

The answer of Jürgen contains the good syntax...

it's not `awk` ignored it - `awk` reads that as one regex's boolean outcome ( 1 / 0 ) , then `numerically minus` a variable named `F`, then string concat with a single colon (`:`), which means the total pattern yielded true because it's a non-empty string, thus `$1` split by default space gets printed — RARE Kpop Manifesto, Jul 09 '22 at 15:25

RARE Kpop Manifesto · Answer 9 · 2022-07-09T15:18:41.913

-1

echo "1: " | "456:abc:515:xyz "

awk -F: NF=/1/

      1    |  456

UPDATE : realizing how verbose my previous answer was

edited Jul 09 '22 at 15:18

answered Jun 09 '22 at 12:44

RARE Kpop Manifesto

2,453
3
11

How can I use ":" as an AWK field separator?

9 Answers9

Linked

Related