-1

I have a script that is fired off 50 times simultaneously executing sed -i to make edits in a single text file. Each script is trying to edit the same file and I suspect there is a conflict since not all the edits are saving.

Sed is building a temp file then amalgamating the changes to original file.

How can I avoid this conflict?

Giancarlo D
  • 35
  • 3
  • 7
    You can't run multiple edits to the same file in parallel with sed (or most other tools). Don't even try, it's just doomed to failure. Either run them sequentially, or (better) have a single instance of sed make all of the changes in a single pass. – Gordon Davisson Apr 29 '23 at 04:09
  • See also https://stackoverflow.com/questions/7657647/combining-two-sed-commands – tripleee Apr 29 '23 at 18:02

1 Answers1

2

As Gordon commented, you can't edit a file simultaneously. But fear not; there are ways to edit it from multiple sources, or to do multiple independent edits.

Using sed's "replace all"

If all you're trying to do is replace one pattern multiple times, sed has a builtin flag ('g') for it.

Input: Mary had a little lamb and a little cat

Edit: Change all occurrences of "a little" with "a big"

Run: sed 's/a little/a big/g'

Using multiple expressions

You can do multiple edits from the same sed. It can take an unlimited(ish) number of -e arguments, each specifying an independent expression, which is run after all the previous expressions.

Input: Mary had a little lamb and a little cat

Edit: Change "lamb" to "tarantula" and "cat" to "hamster"

Run: sed -e 's/lamb/tarantula/' -e 's/cat/hamster/'

Using file locking

For cases where you are unable to form one sed expression to rule them all, you could run multiple sed instances in different shells. This is the least recommended option because:

  • it will still do the edits sequentially, as only one sed can safely run at a time
  • the order of operations is not guaranteed, so if two expressions overlap, it will have undefined results (e.g. replace "little" with "big" and also replace "big" with "huge").
INOUT="/path/to/my/file"
{
    flock --timeout 300 -x $auto_descriptor
    sed  -i 's/Mary/Jane/' "$INOUT"
} {auto_descriptor}>>"$INOUT"

(bash 4.1+ needed for the auto_descriptor part)

Sir Athos
  • 9,403
  • 2
  • 22
  • 23
  • 4
    AND, multiple expressions can be chained together seperated by semicolon (`;`) chars OR on separate lines in a the shell script (where the whole script is enclosed in single quotes) OR where each expression is on a separate line and the expessions are in a file as a `sed` script, and invoked as `sed -f changer.sed data.txt > newData.txt`. Love the idea of a huge hamster! Good luck to all. – shellter Apr 29 '23 at 22:26