unix find command error

Question

My command looks like this:

P8.1 ~basicsys/win15/ex8/d1 cd 3 D A X F1 B

So I have three parameters: dir (holds the directory) str (a string im looking for) num (line number)

What I need to do is to go over the files and check if the str is appearing in line num.

And I need to print something like :

[File] [number of times str appeared in line number num somewhere in the tree of the directory].

output:

A 1
B 3
D 2
F1 1
X 0

From my debugging i found that I have problem with the line that im preforming the find command (in the while).

This is my code:

dir=$1
shift
str=$1
shift
num=$1
shift
echo 'head -$num | tail -1' >| getLine
echo -n "" >| out
chmod +x getLine

while [ $# -gt 0 ]
 do
  echo -n $1 " " >> out
  find $dir -type f -name $1 -exec getLine {} \; >| tmp
  egrep -c $str tmp >> out
shift
done
sort out

maybe the problem is also at echo 'head -$num | tail -1'

pleaseeeee help :/ thanks!!!

Replace quotes `'` by double-quotes `"` in order to see you variable `$num` expanded! — F. Hauri - Give Up GitHub, Jan 10 '15 at 15:14
@F.Hauri thanks!!! add it as an answer and ill accept it bro ! — nick shmick, Jan 10 '15 at 15:19
Something like: `find $dir -type f -name $1 -exec sed $num'q;d' {} \; ...` — F. Hauri - Give Up GitHub, Jan 10 '15 at 15:21
What could contain `$str` variable and "*one line*" (what look `out` like?) — F. Hauri - Give Up GitHub, Jan 10 '15 at 15:48

F. Hauri - Give Up GitHub · Accepted Answer · 2015-01-10T15:58:03.040

1

First

You have to replace quotes ' by double-quotes " in order to see you variable $num expanded!

Instead of using `head | tail`, try `sed`:

find $dir -type f -name $1 -exec sed $num'q;d' {} \;

My purpose (using pure bash):

Without forks and temporary files.

#!/bin/bash

dir=$1 str=$2 num=$3
shift 3

for name ;do
    count=0
    while read file ;do
        mapfile -s $[num-1] -tn 1 lineNum <"$file"
        [ "$lineNum" ] && [ -z "${lineNum//*$str*}" ] && ((count++))
    done < <(find $dir -type f -name $name -print)
    echo $name $count
done |
    sort

edited Jan 10 '15 at 15:58

answered Jan 10 '15 at 15:26

F. Hauri - Give Up GitHub

64,122
17
116
137

`$num` is user-controlled. If the user doesn't enter a number, this can break in all sorts of surprising ways. It's bad enough to substitute shell variables unsanitized into sed code, but this is outside even a double-quoted shell string. – Wintermute Jan 10 '15 at 15:36
@Wintermute You're right, but as it could be better to sanitize entries before processing, this could work with same syntax. Of course, `sed "${num}q;d"` will work too, but for sample, if user hit `w/etc/passwd;2` for `$num`, using double quotes will just help this attack attempt... – F. Hauri - Give Up GitHub Jan 10 '15 at 16:03
My general approach/suggestion is: When you're tempted to put shell variables into sed code, try to do it with `awk -v var="$var"` instead; it'll just error out on type mismatch, and you're about as safe as you can be in a scripting language. Sed may be sexier than awk, but user-modifiable code is icky no matter which way you look at it, and many/most things sed can do are easily translatable to awk. The missing backrefs are sometimes vexing, but they're not needed for this. In this case `awk -v line="$num" 'FNR == line'` does the same as `sed "$num q;d"`. – Wintermute Jan 10 '15 at 16:08
@Wintermute `sed` is a lot quicker than `awk` for many calls, it could become important. – F. Hauri - Give Up GitHub Jan 10 '15 at 16:18
Discounting special circumstances, correctness is more important than speed. And quite honestly, when performance becomes critical, the trick is to stop writing shell scripts. – Wintermute Jan 10 '15 at 16:24

Wintermute · Answer 2 · 2015-01-11T13:48:04.320

The trouble, I think, is that your getLine script does not use its parameters. It might work with

# also, variable expansion does not work in '' strings, like the comments noted.
echo "head -\"$num\" \"$1\" | tail -1" >| getLine

However, this approach strikes me as fairly ugly. I'd do it with awk like this:

#!/bin/sh

dir="$1"
shift
line="$1"
shift
str="$1"
shift

for file in "$@"; do 
    find "$dir" -type f -name "$file" -exec awk -v f="$file" -v l="$line" -v s="$str" -v c=0 'FNR == l { if(index($0, s)) { ++c } nextfile } END { print f " " c }' '{}' +
done

There are two key components to this: One is that the + in the find call makes it pass all matching files to the awk command in one go (for precise semantics see the find man page). The other is the awk script:

FNR == l {           # if the line is the lth in the file we're processing
  if(index($0, s)) { # and s is a substring of the line
    ++c              # increase the counter
  }
  nextfile           # short-circut to next file. This may fail with old versions
                     # of awk; it was introduced to the POSIX standard in 2012.
                     # It can be removed in that case, although that will make
                     # the script run more slowly (because it'll scan every file
                     # to the end)
}
END {                # at the very end
  print f " " c      # print the results.
}

Here l, s, and f are set through the -v options to the user-defined values in the main script..

unix find command error

2 Answers2

First

Instead of using head | tail, try sed:

My purpose (using pure bash):

Instead of using `head | tail`, try `sed`: