0

I have a folder that contains 300 files. I would like to remove lines from the files if $2<=25. How can I remove the lines directly from the files(in place editing)? My code is shown below.

awk '{ for (i=1; i<=NF; i++) { if ($2 <= 25) next } print }' < *

ads 54.5    18  15.3    39.2
bdy 18.5    21  1.5     17.0
cst 36.8    22  27.7    9.1
hst 40.2    25  16.2    24.0
ads 18.9    41  5.0     13.2
bdy 20.5    45  67.0    19.0
vikram
  • 3
  • 2
  • See: [Delete all lines where column 1 has specific numeric entries](http://stackoverflow.com/questions/22697636/delete-all-lines-where-column-1-has-specific-numeric-entries) and [awk save modifications inplace](http://stackoverflow.com/questions/16529716/awk-save-modifications-inplace) – Cyrus May 15 '16 at 14:03
  • 1
    I highly recommend you read the book Effective Awk programming, 4th Edition, by Arnold Robbins before trying to do any more text manipulation as you seem very confused about awk syntax (e.g. looping through all fields testing just $2 repeated each time) and invocation (e.g. trying to use input redirection from a list of files and hoping to somehow change values in a file that awk doesn't know exists courtesy of that redirection). – Ed Morton May 15 '16 at 15:55

1 Answers1

2

You're script is doing too much. The cleanest solution, to my mind, inverts the condition:

awk '{ if ($2 > 25) print }'

or even:

awk '$2 > 25'

If you don't want to invert the condition, then:

awk '{ if ($2 <= 25) next; print }'

There's no need to iterate over all the fields.

Not even GNU awk supports 'in situ' file modification. You have to write the result to a temporary file, and then copy or move the temporary back over the original. (Copy preserves hard links and permissions; move breaks links and can modify owner and permissions. You need to decide whether that's a concern.)

Thanks to Ed Morton for pointing out that GNU Awk 4.x does have a mechanism to edit files 'in situ', in part because he campaigned to get it added.

The command line help won't tell you that GNU Awk 4.x supports in-place modification of files, but if you find the right part of the manual (Extension sample: inplace — which is mis-titled from my perspective; it isn't just a sample because it is a distributed extension) then you can find out that there is an extension that makes GNU Awk overwrite regular files specified on its command line.

gawk -i inplace '{ if ($2 > 25) print }' file1 …

or even:

gawk -i inplace '$2 > 25' file1 …

Note that experimentation shows that it is quite happy to modify read-only files in situ. This is consistent with sed (both GNU and BSD (Mac OS X) sub-species); they also modify read-only files in situ without warning — and preserve the permissions on the file but break any hard links.

Your script uses awk '…' < *; that is a peculiar way of ignoring the first file in your directory unless it is the only file in the directory (it is used for standard input, but if there's more than one file in the directory, standard input is ignored). You need to use just *, not < *.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278