Copy code between two lines from one file and overwrite between same two lines in another file

Question

I am trying to copy codes between two specific lines from one file and paste it between same two corresponding lines in another file.

For example I have two files test.sv_old and test.sv. I want to copy code from test.sv_old file to test.sv between below two lines

Line 1:"//Start of functional specification here"

Line2:"// Outputs set to 0 if no supply. Uncomment as needed."

Here is the content of test.sv_old file:

`include "def.sv"
/PRIMARY  
/SECONDARY  
/TERTIARY  
/UNASSIGNED
module abc (  );
we want to see this too
//Start of functional specification here


//Functional cell instantiation 
abc_real Inst0  (.z1(int_z1), 
    .z2(int_z2), 
    .a1(reg_a1));

// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // abc
`include "def.sv"
/PRIMARY  
/SECONDARY  
/TERTIARY  
/UNASSIGNED
module xyz (  );
//Start of functional specification here


//Functional cell instantiation 
xyz_real Inst0  (.y1(int_y1), 
    .y2(int_y2), 
    .a1(reg_a1));

// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // xyz
`include "def.sv"
/PRIMARY  
/SECONDARY  
/TERTIARY  
/UNASSIGNED
module lmn (  );
//Start of functional specification here


//Functional cell instantiation 
lmn_real Inst0  (.x1(int_x1), 
    .x2(int_x2), 
    .a1(reg_a1));

// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // lmn

Here is my test.sv file:

`include "def.sv"
//PRIMARY  
//SECONDARY  
//TERTIARY 
//UNASSIGNED 
module abc (  );
keep this code untouched
no change needed here
//Start of functional specification here


//Functional cell instantiation 
some garbage
here 
just replace this

// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // abc
`include "def.sv"
//PRIMARY  
//SECONDARY  
//TERTIARY  
//UNASSIGNED 
module xyz (  );
keep this as it is
input a1;
//Start of functional specification here


//Functional cell instantiation 
some garbage
here and there
why not just replace this


// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // xyz
`include "def.sv"
//PRIMARY  
//SECONDARY 
//TERTIARY 
//UNASSIGNED 
module lmn (  );
keep this as it is
input a1;
//Start of functional specification here


//Functional cell instantiation 
some garbage
here and there
why not just replace this


// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // lmn

I have tried below code but it doesn't give me exact output I need:

import sys,re,os

rf_SVFile=open(sys.argv[1],"r")

wtstring = ""
wtindex = 0
copy = False
write = False
print("Copying instantiation code from {} to new SV file {}".format(rf_SVFile.name,sys.argv[2]))
for vline in rf_SVFile:
    if vline.strip() == "//Start of functional specification here" and copy == False:
        copy = True
    elif vline.strip() == "// Outputs set to 0 if no supply. Uncomment as needed.":
        copy = False
    elif copy:
        wtstring = wtstring + vline  # wtstring has the functional code between two lines which you want to write to .sv file

with open(sys.argv[2], "r+") as wf_NewSVFile:
    insert = False
    contents = wf_NewSVFile.readlines()
    for index, svline in enumerate(contents):
        if svline.strip() == "// Outputs set to 0 if no supply. Uncomment as needed.":
            wtindex = index
            insert = True
            break
    contents.insert(wtindex,wtstring)  # contents has complete code in list format, instantantiation code is copied from SV file to new SV File
    stringContents = "".join(contents)  # convert list into string in order to write it to .sv file
    if insert:
        wf_NewSVFile.seek(0, 0)
        wf_NewSVFile.write(str(stringContents))
    else:
        print(
            'Warning: No "/ Outputs set to 0 if no supply. Uncomment as needed." line found in {}, hence code is not being copied to new SV file',NewSVFile)

and here is the modified test.sv file generated by above code:

`include "def.sv"
//PRIMARY  
//SECONDARY  
//TERTIARY 
//UNASSIGNED 
module abc (  );
keep this code untouched
no change needed here
//Start of functional specification here


//Functional cell instantiation 
some garbage
here 
just replace this



//Functional cell instantiation 
abc_real Inst0  (.z1(int_z1), 
    .z2(int_z2), 
    .a1(reg_a1));



//Functional cell instantiation 
xyz_real Inst0  (.y1(int_y1), 
    .y2(int_y2), 
    .a1(reg_a1));



//Functional cell instantiation 
lmn_real Inst0  (.x1(int_x1), 
    .x2(int_x2), 
    .a1(reg_a1));

// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // abc
`include "def.sv"
//PRIMARY  
//SECONDARY  
//TERTIARY  
//UNASSIGNED 
module xyz (  );
keep this as it is
input a1;
//Start of functional specification here


//Functional cell instantiation 
some garbage
here and there
why not just replace this


// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // xyz
`include "def.sv"
//PRIMARY  
//SECONDARY 
//TERTIARY 
//UNASSIGNED 
module lmn (  );
keep this as it is
input a1;
//Start of functional specification here


//Functional cell instantiation 
some garbage
here and there
why not just replace this


// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // lmn

Can anyone explain, what I am doing wrong? Thanks.

how about read() (not realine(), both files, string.split() on both marker comments, giving you 3 chunks each. file out becomes f2p1, f1p2, f2p3 — JL Peyret, Oct 16 '18 at 15:19
Stop it. Just stop it. You're apparently trying to implement something like source control uses to generate and apply patches. Look for existing tools that do this and use them. Or if your problem is more typical, you might be able to use a **template** that you can expand based on some configuration parameters. Whatever this is, it's [bad code](https://i.stack.imgur.com/a0bTh.jpg). — jpmc26, Oct 16 '18 at 16:40

score 2 · Answer 1 · answered Oct 16 '18 at 17:53

2

Using some keywords to create indexes and then joining the slices should do the trick.

with open('test.sv') as f:
    content = f.readlines()

with open('test_old.sv') as f:
    content_2 = f.readlines()

cp_s = [i for i, v in enumerate(content_2) if 'Functional' in v]
cp_end = [i for i, v in enumerate(content_2) if 'Outputs' in v]
dest_s = [i for i, v in enumerate(content) if 'Functional' in v]
dest_end = [i for i, v in enumerate(content) if 'Outputs' in v]

new = content[:dest_s[0]] + content_2[cp_s[0]: cp_end[0]] + content[dest_end[0]: dest_s[1]] +
content_2[cp_s[1]:]

with open('fixed.sv', 'w') as f:
    f.write(''.join(new))

Output:

chrx@chrx:~/python/stackoverflow/10.11$ cat fixed.sv 
module abc (  );
keep this code untouched
no change needed here
//Start of functional specification here


//Functional cell instantiation 
abc_real Inst0  (.z1(int_z1), 
    .z2(int_z2), 
    .a1(reg_a1));

// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // abc

module xyz (  );
keep this as it is
input a1;
//Start of functional specification here


//Functional cell instantiation 
xyz_real Inst0  (.z1(int_z1), 
    .z2(int_z2), 
    .a1(reg_a1));


// Outputs set to 0 if no supply. Uncomment as needed.
endmodule // xyz

answered Oct 16 '18 at 17:53

vash_the_stampede

4,590
1
8
20

This is great, much simplified and works like a charm. – sanforyou Oct 16 '18 at 18:11
@sanforyou glad to help :) This was actually pretty fun to slice up got me paying attention – vash_the_stampede Oct 16 '18 at 18:12
how can I make it more generic.. like if I have more than two modules? It breaks if I add a third module and just copies everything form third module as it is. – sanforyou Oct 16 '18 at 18:25
@sanforyou well this involves slicing both files and merging them, if you had a third file you would have to slice that accordingly with whatever file you would want to add that content to – vash_the_stampede Oct 16 '18 at 18:27
No, I have only two files but when I add third module (module lmn) then it doesn't work. I have updated my main post with new test.sv_old file. – sanforyou Oct 16 '18 at 18:37
@sanforyou I see yeah, you will just have to slice that into it, I'll update for that here in a minute, taking a 15 min break – vash_the_stampede Oct 16 '18 at 18:40

score 1 · Answer 2 · answered Oct 16 '18 at 07:24

Question: Can anyone explain, what I am doing wrong?

Your code is far away from Complete.
I would recommend the following logic:

Read test.sv_old and test.sv files line by line
From every module up to endmodule make a list of dict {<module name>:<module body>}
With these Conditions:
From test.sv_old read only from //Functional up to //Output
From test.sv read all except from //Functional up to //Output, keeping //Functional as a placeholder.
Loop the list of dict from test.sv
1. Write from dict {<module name>:<module body>} line by line
2. At the line //Functional switch to dict from test.sv_old and write the whole <module body>
3. Continue writing remaining <module body> from test.sv

JL Peyret · Answer 3 · 2018-10-16T17:57:32.817

This seems to do what you wanted. i.e. when I diff your desired test.sv to test_so79.new.txt they're equal. I left some debug stuff, you might need it ;-) And the regex splitting came from In Python, how do I split a string and keep the separators?

import re
import pdb

def chunker_module(module_):

    # might need to make them more specific, but to lazy to regex-escape
    # all your stuff
    marker1 = "//Start.+$"
    marker2 = "//\sOutputs.+$"

    patre = re.compile("(%s|%s)" % (marker1, marker2), re.MULTILINE)

    res = patre.split(module_)
    try:
        assert len(res) == 5
    except (Exception,) as e:
        pdb.set_trace()
        raise

    head, tgt, tail = (res[0] + res[1], res[2], res[3] + res[4])

    return head, tgt, tail

def chunk_file(fnp):

    patre = re.compile("(module\s.+;$)", re.MULTILINE)

    with open(fnp) as fi:
        code = fi.read()

    splits_ = patre.split(code)


    modules = []

    #hmmm, wonder if the 1+2, 3, 4+5 approach would work here too...
    current = ""
    for item in splits_:
        if patre.search(item):
            modules.append(current)
            current = item
        else:
            current += item
    modules.append(current)

    # def debug_():
    #     for ix, mo in enumerate(modules):
    #         print("%s:\n%s" % (ix,mo))

    # debug_()
    # pdb.set_trace()

    # print(modules)
    return modules

modules_old = chunk_file("test_so79.old.txt")
modules_new = chunk_file("test_so79.txt")

lines_out = []

for mo, mn in zip(modules_old, modules_new):
    #actually, if mo/mn doesn't start with your //module marker
    #you might to append mn to lines_out -- it's from your new
    #file but doesnt need processing
    if not mo:
        continue

    _, keep, _ = chunker_module(mo)
    p1, _, p3 = chunker_module(mn)

    # def debug_():
    #     print(p1, keep, p3)
    # pdb.set_trace()
    # debug_()


    lines_out.extend([p1, keep, p3])

with open("test_so79.new.txt", "w") as fo:
    for line in lines_out:
        fo.write("%s\n" % (line))

To elaborate on my remark about the zip lockstep constraints, if old's module sequence was abc, xyz, and new's was xyz,abc, then you would have to treat things a little differently.

This should get you started:

di_old_module = dict([(module.name,module) for module in modules_old])
....
for mn in modules_new:
   mo = di_old_module[mn.name]
   _, keep, _ = chunker_module(mo)
   p1, _, p3 = chunker_module(mn)

Thanks for your response. This works great. I also like your logic of splitting both files into 3 parts and then pick f2p1, f1p2, f2p3. Could you elaborate a bit on for loop using zip function? — sanforyou, Oct 16 '18 at 17:22
you need to read up on zip. but basically it assumes that your 2 streams of chunks are in lockstep. If they start diverging, i.e. old[n] doesn’t match new[n] then this wont work. — JL Peyret, Oct 16 '18 at 17:32
Ok thanks. Also can you explain the line "assert len(res) == 5" , why are you comparing patre.split(module_) to 5? — sanforyou, Oct 16 '18 at 17:56
You need to make sure that your regex splits are behaving as expected. Most of the complexity I found was stitching the regex.splits back together and you need to stay on the ball on what they're doing. I initially thought this split was going to return 3 items. Looking at it, I saw five and saw that I could effect the required behavior by 1+2, 3, 4+5. But, if there was 4 or 6 items then I would have to adjust the code so the `assert` throws an exception if it was not 5. — JL Peyret, Oct 16 '18 at 18:01
Strange it breaks and len(res) is just 1 when I add some extra lines. I tried to paly around with module pattern but couldn't get it working. Can you try it out with test.sv and test.sv_old in my original post one last time? — sanforyou, Oct 16 '18 at 23:54
i have updated test.sv and test.sv_old in my original post for you — sanforyou, Oct 17 '18 at 00:15

Copy code between two lines from one file and overwrite between same two lines in another file

3 Answers3