Search equality in a certain field with AWK

Question

I am trying to get the name out of /etc/passwd using awk to search only in the 5th field of every row, and then to cut some part of that line and print it out.

This is what I wrote but it doesn't seems to work:

for iter in "$@";
do cat /etc/passwd | awk -F ":" '$5==$iter' | cut -d":" -f6;
done;

concerning the delimiter syntax, everything should be fine I guess? so my problem is in the $5==$iter, I assume. How can I change that $5==$iter to - if the 5th field of that row contains my $iter var, then cut and so on..

Sorry for the ignorance, I am a beginner :) Thanks in advance.

You need to pass `iter` into awk; right now it's only a bash variable and not an awk variable. — Charles Duffy, Sep 12 '21 at 21:38
See [How do I use shell variables in an awk script?](https://stackoverflow.com/questions/19075671/how-do-i-use-shell-variables-in-an-awk-script) -- I'd argue that the questions are outright duplicative of each other. — Charles Duffy, Sep 12 '21 at 21:38
(Also, consider taking out the `cut`; it makes more sense to make your awk script itself print the specific field you want). — Charles Duffy, Sep 12 '21 at 21:39
(also, in general, `cat somefile | someprogram` should _always_ be `someprogram — Charles Duffy, Sep 12 '21 at 21:40
Though it would be better to only read `/etc/passwd` _once_, instead of rereading it once per item you want to search for; especially if the number of items to search for is large. — Charles Duffy, Sep 12 '21 at 21:41
could you update the question with an example of what's typically in `"$@"` as well as an indication of what you could expect as a 'maximum/longest' example? I'm wondering if there could be a solution that doesn't rely on a `bash` looping construct — markp-fuso, Sep 12 '21 at 22:22

score 1 · Accepted Answer · answered Sep 12 '21 at 21:40

1

See How do I use shell variables in an awk script?

-v should be used to pass shell variables into awk. Also, there's no reason to use either cat or cut here:

for iter in "$@"; do
  awk -F: -v iter="$iter" '$5==iter { print $6 }' </etc/passwd
done

answered Sep 12 '21 at 21:40

Charles Duffy

280,126
43
390
441

I rewrote your snippet in my post to only invoke awk once as you suggested. – Mark Sep 13 '21 at 01:49

Mark · Answer 2 · 2021-09-13T14:32:32.343

1

As Charles Duffy commented, your code would be more efficient if it didn't need to read /etc/passwd every pass. And while this particular loop probably doesn't need to be optimized (after all, /etc/passwd is typically not that long and most OS's would cache the file anyway after the first read), it would be interesting to see an awk script read the file only once.

That said, here's another implementation where awk is only invoked once:

printf "%s\n" "$@" | awk -F: '
  NR == FNR { etc_passwd[ $5 ] = $6;  next } 
            { print $0 , etc_passwd[ $0 ]   }
' /etc/passwd  /dev/stdin

The NR == FNR condition is an idiom that causes its associated command only to be executed for the first file in the list of files that follows the awk script (that is, for the reading of /etc/passwd).

edited Sep 13 '21 at 14:32

answered Sep 13 '21 at 01:47

Mark

4,249
1
18
27

Consider `printf '%s\n' "$@" | awk ...` as a slight optimization. (If you aren't going to do that, I'd suggest quoting `"$iter"` in the `echo` command to rule out some corner cases). – Charles Duffy Sep 13 '21 at 11:22
Thanks - I always forget about that aspect of printf. – Mark Sep 13 '21 at 14:32

Ronaldo Ferreira de Lima · Answer 3 · 2021-09-13T02:33:21.260

You can also do everything in bash, example:

#!/bin/bash

declare -A passwd               # declare a associative array

# build the associative array "passwd" with the
# 5th field as a "key" and 6th field as "value"
while IFS=$':\n' read -a line; do      # emulate awk to extract fields
    [[ -n "${line[4]}" ]]  || continue # avoid blank "keys"
    passwd["${line[4]}"]=${line[5]}    # in bash, arrays starting in "0"
done < /etc/passwd

for iter in "$@"; do
    if [ ${passwd[$iter] + 'x'} ]; then
        echo ${passwd[$iter]}
    fi
done

(This version doesn't get into accout múltiples values for 5th field)

here is a better version that can handle blank values as well, ike./script.sh '':

while IFS=$':\n' read -a line; do
    for iter in "$@"; do
        if [ "$iter" == "${line[4]}" ]; then
            echo ${line[5]}
            continue
        fi
    done
done < /etc/passwd

A pure awk solution could be:

#!/usr/bin/awk -f

BEGIN {
    FS = ":"
    for ( i = 1; i < ARGC; i++ ) {
        args[ARGV[i]] = 1
        delete ARGV[i]
    }
    ARGV[1] = "/etc/passwd"
}
($5 in args) { print $6 }

and you could call as ./script.awk -f 'param1' 'param2'.

I don't usually see spaces used in parameter expansions such as `${passwd[$iter] + x}` except where they're necessary to disambiguate -- is the a source for the convention choice here? Indeed, it doesn't appear to work everywhere; I get a `bash: ${passwd[test] + x}: bad substitution` in bash 5.1.8, whereas `${passwd[test]+x}` behaves as-intended. — Charles Duffy, Sep 13 '21 at 11:19
(that first bash example could also use more aggressive quoting; it currently can fall afoul of [I just assigned a variable, but `echo $variable` shows something different!](https://stackoverflow.com/q/29378566/14122) — Charles Duffy, Sep 13 '21 at 11:21

Search equality in a certain field with AWK

3 Answers3