172

I am learning awk and I would like to know if there is an option to write changes to file, similar to sed where I would use -i option to save modifications to a file.

I do understand that I could use redirection to write changes. However is there an option in awk to do that?

codeforester
  • 39,467
  • 16
  • 112
  • 140
Deano
  • 11,582
  • 18
  • 69
  • 119
  • In case anyone wants to have inplace save with NON GNU `awk` could use following link too https://stackoverflow.com/questions/59243104/save-modifications-in-place-with-non-gnu-awk fyi please. – RavinderSingh13 Aug 25 '20 at 17:07
  • Everyone should read https://backreference.org/2011/01/29/in-place-editing-of-files/ which addresses the topic of "inplace" editing in general, not just using awk. – Ed Morton May 12 '22 at 00:06

7 Answers7

182

In GNU Awk 4.1.0 (released 2013) and later, it has the option of "inplace" file editing:

[...] The "inplace" extension, built using the new facility, can be used to simulate the GNU "sed -i" feature. [...]

Example usage:

$ gawk -i inplace '{ gsub(/foo/, "bar") }; { print }' file1 file2 file3

To keep the backup:

$ gawk -i inplace -v INPLACE_SUFFIX=.bak '{ gsub(/foo/, "bar") }
> { print }' file1 file2 file3
jeremysprofile
  • 10,028
  • 4
  • 33
  • 53
lind
  • 2,159
  • 1
  • 12
  • 5
  • Looks like the option may have been removed? With 4.1.3, I have "-i includefile --include=includefile" – Keith Hughitt Jun 18 '16 at 11:46
  • 5
    @Keith I had the same question. I just tried it and it works on my 4.1.3. `inplace` is actually a library included with `gawk` according to [iiSeymour's answer](http://stackoverflow.com/a/16529730/2877364), so `inplace` is something that can be included as an `includefile`. – cxw Jun 23 '16 at 20:16
  • 1
    An important caveat here: the 'seen' array will fill up with duplicate lines from ALL the files included in the command. So if each file has e.g. a common header, that will be removed in every file after the first one. If you instead want to treat each file independently, you'll need to do something like for f in *.txt; do gawk -i inplace '!seen[$0]++' "$f"; done – Nick K9 Jan 17 '19 at 18:10
  • 1
    Note: Since GNU Awk 5.x, the `INPLACE_SUFFIX` is now a namespaced variable called `inplace::suffix`. The extension will **fall back** to the old variable, but **not in versions 5.0.0 and 5.0.1** (5.0.1 ships with Ubuntu 20.04 LTS), only since 5.1.0. But you can simply pass both variables to avoid any problems. – Ingo Karkat Apr 08 '22 at 17:43
  • @IngoKarkat Good review, you're right! Your note was useful for me, thanks! – Rogelio Prieto Dec 23 '22 at 16:04
171

Unless you have GNU awk 4.1.0 or later...

You won't have such an option as sed's -i option so instead do:

$ awk '{print $0}' file > tmp && mv tmp file

Note: the -i is not magic, it is also creating a temporary file sed just handles it for you.

As of GNU awk 4.1.0...

GNU awk added this functionality in version 4.1.0 (released 10/05/2013). It is not as straight forwards as just giving the -i option as described in the released notes:

The new -i option (from xgawk) is used for loading awk library files. This differs from -f in that the first non-option argument is treated as a script.

You need to use the bundled inplace.awk include file to invoke the extension properly like so:

$ cat file
123 abc
456 def
789 hij

$ gawk -i inplace '{print $1}' file

$ cat file
123
456
789

The variable INPLACE_SUFFIX can be used to specify the extension for a backup file:

$ gawk -i inplace -v INPLACE_SUFFIX=.bak '{print $1}' file

$ cat file
123
456
789

$ cat file.bak
123 abc
456 def
789 hij

I am happy this feature has been added but to me, the implementation isn't very awkish as the power comes from the conciseness of the language and -i inplace is 8 characters too long i.m.o.

Here is a link to the manual for the official word.

Chris Seymour
  • 83,387
  • 30
  • 160
  • 202
  • Shouldn't your 'first' example be more like: ```awk '{ gsub(/foo/, "bar" ) } ; { print $0 }' file > tmp.txt && mv -v tmp.txt file``` ? – Tony Barganski Oct 03 '18 at 03:28
  • To my surprise, as of April 2019, still at gawk 4.0.2. Don't let anyone tell you such and such version will be available. – John Lunzer Apr 15 '19 at 23:03
  • 2
    Litte shorter `awk '{print $0}' file | sponge file` using `sponge` from `moreutils`. – brablc Mar 22 '20 at 15:44
  • Note that `mv tmp file` is not in general sufficient - the file perms may have changed and must be fixed. `-i inplace` doesn't have this problem. – EML Jan 29 '22 at 13:01
28

just a little hack that works

echo "$(awk '{awk code}' file)" > file
Yuri G.
  • 4,323
  • 1
  • 17
  • 32
18

An alternative is to use sponge:

awk '{print $0}' your_file | sponge your_file

Where you replace '{print $0}' by your awk script and your_file by the name of the file you want to edit in place.

sponge absorbs entirely the input before saving it to the file.

Codoscope
  • 892
  • 1
  • 10
  • 18
  • How standard/portable is sponge? – Thomas Oct 09 '18 at 11:03
  • 3
    `sponge` is part of `moreutils`. So it won't be present by default in most systems. But looks like at least `sponge` itself is portable enough and can be run almost everywhere. – MarSoft Dec 14 '18 at 11:21
  • 3
    The downside of this solution compared to `tee`-based is that `sponge` will read everything to RAM before writing down, hence it will freeze on large files. – MarSoft Dec 14 '18 at 11:22
17

@sudo_O has the right answer.

This can't work: 

someprocess < file > file

The shell performs the redirections before handing control over to someprocess (redirections). The > redirection will truncate the file to zero size (redirecting output). Therefore, by the time someprocess gets launched and wants to read from the file, there is no data for it to read.

Community
  • 1
  • 1
glenn jackman
  • 238,783
  • 38
  • 220
  • 352
6

following won't work

echo $(awk '{awk code}' file) > file

this should work

echo "$(awk '{awk code}' file)" > file
Flowmix Leonsio
  • 91
  • 2
  • 5
4

In case you want an awk-only solution without creating a temporary file and usable with version!=(gawk 4.1.0):

awk '{a[b++]=$0} END {for(c=0;c<=b;c++)print a[c]>ARGV[1]}' file
Hawk
  • 121
  • 1
  • 4