Subtle finding and replacing with grep

Question

I have a bunch of files, all of them in the same directory, with names opening1.py, opening2.py... and all of them have line(s) with 'steps': [6]} ],. What I want to do is to replace 6 by a 9. I have been trying the solution here: How to grep and replace

with a command like this one:

grep -rl "'steps': [6]} ]," opening* | xargs sed -i "s/'steps': [6]} ],/'steps': [9]} ],/g"

but it doesn't work. It says sed: no input files. I don't know if the problem is due to the presence of spaces in the string or ticks '. How can I do it?

Thanks.

P. S.: In case it may help: files opening... are quite big. The string I'm searching is inside a for loop, also a big and apparently horrible one like this:

for num in range(4):
    date_c = date_in
    while date_c <= date_in:
        lat = coordinates[num][0]
        lon = coordinates[num][1]
        df1 = gat.select_grib_df(inputfile,
                    {'indicatorOfParameter': 11},
                    [ {'coord': (lat, lon)],
                    'date': date_c,
                    'steps': [6]} ],
                    ['indicatorOfParameter', 'dataDate', 'dataTime', 'step'])

and I want that to be exactly like before but with 9 in steps instead of 6: Note: sometimes the number in indicatorOfParameter can change.

for num in range(4):
date_c = date_in
while date_c <= date_in:
    lat = coordinates[num][0]
    lon = coordinates[num][1]
    df1 = gat.select_grib_df(inputfile,
                {'indicatorOfParameter': 11},
                [ {'coord': (lat, lon)],
                'date': date_c,
                'steps': [9]} ],
                ['indicatorOfParameter', 'dataDate', 'dataTime', 'step'])

the only change is the number 6 per 9 in the line with the word steps.

Possible duplicate of [How can I avoid “no input files” error from sed?](https://stackoverflow.com/questions/35431718/how-can-i-avoid-no-input-files-error-from-sed) -- this copes with the scenario where your `grep` doesn't find any matches (but as pointed out separately, the `grep` here is useless anyway). — tripleee, Oct 08 '18 at 12:03

Ed Morton · Accepted Answer · 2018-10-08T12:01:49.870

1

This may be what you're looking for:

$ sed 's/\('\''steps'\'': \[\)6\(]} ],\)/\19\2/' file
'steps': [9]} ],

Just run it as:

sed -i 's/\('\''steps'\'': \[\)6\(]} ],\)/\19\2/' opening*

no need for the grep.

Given your updated sample input:

$ sed 's/\('\''steps'\'':[[:space:]]*\[\)6\(][[:space:]]*}[[:space:]]*][[:space:]]*,\)/\19\2/' file
for num in range(4):
    date_c = date_in
    while date_c <= date_in:
        lat = coordinates[num][0]
        lon = coordinates[num][1]
        df1 = gat.select_grib_df(inputfile,
                    {'indicatorOfParameter': 11},
                    [ {'coord': (lat, lon)],
                    'date': date_c,
                    'steps': [9]} ],
                    ['indicatorOfParameter', 'dataDate', 'dataTime', 'step'])

edited Oct 08 '18 at 12:01

answered Oct 08 '18 at 11:42

Ed Morton

188,023
17
78
185

1

Thanks, I have edited my question to show you the full context – David Oct 08 '18 at 11:51
1

Thank you, it works. I barely know `sed`, so I am not asking you for the details, but may I ask what is the difference between the `-i` option and without it, and how should I modify your instance in case I want `12` or `15` instead of `9`? – David Oct 08 '18 at 12:32
1

sed by default prints its output to stdout. When you add -i it modifies the input file instead. To use 12 instead of 9, just change `9` to `12` in the script. – Ed Morton Oct 08 '18 at 12:41

tripleee · Answer 2 · 2018-10-08T12:05:50.040

Mac and BSD sed require an argument to the -i option. If you require the script to be portable to Linux/GNU sed, perhaps refactor it into Perl.

sed -i '' "s/'steps': \[6\]} \],/'steps': [9]} ],/" opening*.py

In Perl:

perl -pi -e "s/'steps': \[6\]\} \],/'steps': [9]} ],/" opening*.py

Notice how neither of these use grep -- if an input file doesn't contain a match, it will be unmodified (though some implementations will blindly rewrite the file anyway and update the time stamp in the process).

Also, notice how the square brackets (and in Perl, the curly bracket) need to be escaped in the regex, and how the /g flag is unnecessary if you don't expect multiple matches on a single line. (Some regex engines will tolerate an unescaped ] just fine as long as it doesn't have an unescaped [ before it.)

But a much better solution is to refactor your Python script so that you can pass in the steps and any other parameter(s) (opening?) as command-line arguments.

import sys

if len(sys.argv) == 1:
    # Should properly print to stderr, use logging?
    print('Syntax: opening.py <steps>')
    exit 1
steps_param = int(sys.argv[1])

for num in range(4):
    date_c = date_in
    while date_c <= date_in:
        lat = coordinates[num][0]
        lon = coordinates[num][1]
        df1 = gat.select_grib_df(
            inputfile,
            # indicatorOfParameter should apparently be refactored as well
            {'indicatorOfParameter': 11},
             [ {'coord': (lat, lon)],
                'date': date_c,
                'steps': [steps_param]} ],
             ['indicatorOfParameter', 'dataDate', 'dataTime', 'step'])

Thanks, your Python refactoring solution is quite nice, too, but take into consideration that you need to rewrite that in every line of the `opening...` files, so in the end I think a `sed` is necessary. — David, Oct 08 '18 at 12:10
Ideally you would refactor all of `opening*.py` into a single script which you run with different parameters. Doing anything more than trivial substitutions with `sed` is going to be hurtful and error-prone. Maybe consider generating the variations from a single template file or something if you can't reduce this to a single script. — tripleee, Oct 08 '18 at 12:10

Subtle finding and replacing with grep

2 Answers2