How to do multiline search and replace with a script?

Question

I'm trying to replace every multiline import inside a Python source file.. So, the source goes like

from XXX import (
   AAA,
   BBB,
)
from YYY import (
   CCC,
   DDD,
   EEE,
   ...
)
...other instructions...

and I'd like to get something like

from XXX import AAA, BBB
from YYY import CCC, DDD, EEE, ...
...other instructions...

I tried to use sed but it looks like it doesn't support non-greedy matching of the closing parenthesis, so it "eats" the second import.. :(
Any hint? Is this impossible with sed? Should I try with another tool?

is there always 2 tokens after the 'import' or are they variable? — Gishu, Dec 15 '08 at 11:12
I was wondering if this could be done via a single reg-ex :) — Gishu, Dec 15 '08 at 11:50

score 2 · Answer 1 · answered Dec 22 '11 at 16:40

2

This might work for you:

sed '/^from/,/^)/{H;//{x;/)/{s/[\n()]//g;s/  */ /g;s/,$//;p;x}};d}' source
from XXX import AAA, BBB
from YYY import CCC, DDD, EEE, ...
...other instructions...

answered Dec 22 '11 at 16:40

potong

55,640
6
51
83

score 1 · Accepted Answer · answered Dec 15 '08 at 11:04

1

Ummm... what's wrong with Python?

lineIter= iter(aFile)
for aLine in lineIter:
    if aLine.startswith("import"):
        if aLine.endswith("("):
            for aModule in lineIter:
                if aModule.endwith(")"):
                    break
                print "import", aModule.strip()
        else:
            print aLine.stri()
    else:
        print aLine.strip()

answered Dec 15 '08 at 11:04

S.Lott

384,516
81
508
779

Not every system has it installed? A lot of the machines I work on certainly don't. – GodEater Dec 15 '08 at 11:41
Bryan, since he wants to modify a python source file, bets are that he has access to some machine with python installed :) – Svante Dec 15 '08 at 11:45
Python is perfectly fine :) I was trying with sed because it looked like the right tool for the task :) – Joril Dec 15 '08 at 12:35
@Joril: sed's hard to use (as you've noted). Therefore, it isn't really the right tool for task, is it? – S.Lott Dec 15 '08 at 13:52
Well, it would have been if it had support for non-greedy matching :) But I didn't know it X-) – Joril Dec 15 '08 at 14:03
@Joril: I disagree -- if the non-greedy matching isn't obvious, it's too hard to use. And, for most tasks like this, it doesn't really add any value to mash the process down to a few cryptic lines of incomprehensible sed. – S.Lott Dec 15 '08 at 16:11
I see your point.. sed expressions are often somewhat "write-only" :D But I still find it very handy for small tasks (not like this one of course). Thanks for sharing your thoughts! :) – Joril Dec 15 '08 at 19:36
http://stackoverflow.com/questions/366980/what-are-the-differences-between-perl-python-awk-and-sed#367082 -- can't see the value in sed. Used to use sed. Gave it up because Python seemed to always be better. – S.Lott Dec 15 '08 at 19:41

score 1 · Answer 3 · answered Dec 15 '08 at 13:06

For posterity, here is a somewhat polished version of S.Lott's script (I'd have posted it as a comment, but it's too long ^^; ).. This version preserves indentation and produces a result closer to my example.

lineIter=iter(aFile)
for aLine in lineIter:
    s = aLine.strip()
    if s.startswith("from ") and s.endswith("("):
        complete = s[:-1]
        for aModule in lineIter:
            m = aModule.strip()
            if m.endswith(")"):
                break
            complete += m.strip()
        print complete
    else:
        print aLine,

How to do multiline search and replace with a script?

3 Answers3