python code to search a text string in multiple text file

Question

I have multiple text files in a folder say "configs", I want to search a particular text "-cfg" in each file and copy the data after -cfg from opening to closing of inverted commas ("data"). This result should be updated in another text file "result.txt" with filename, test name and the config for each file.

NOTE: Each file can have multiple "cfg" in separate line along with test name related to that configuration.

E.g: cube_demo -cfg "RGB 888; MODE 3"

My approach is to open each text file one at a time and find the pattern, then store the required result into a buffer. Later, copy the entire result into a new file.

I came across Python and looks like it's easy to do it in Python. Still learning python and trying to figure out how to do it. Please help. Thanks.

I know how to open the file and iterate over each line to search for a particular string:

import re
search_term = "Cfg\s(\".*\")"           // Not sure, if it's correct
ifile = open("testlist.csv", "r")
ofile = open("result.txt", "w")
searchlines = ifile.readlines()
for line in searchlines:
    if search_term in line:
        if re.search(search_term, line):
            ofile.write(\1)              
// trying to get string with the \number special sequence
ifile.close()
ofile.close()

But this gives me the complete line, I could not find how to use regular expression to get only the "data" and how to iterate over files in the folder to search the text.

Answer 1

Not quite there yet...

import re
search_term = "Cfg\s(\".*\")"           // Not sure, if it's correct

"//" is not a valid comment marker, you want "#"

wrt/ your regexp, you want (from your specs) : 'cfg', followed by one or more space, followed by any text between double quotes, stopping at the first closing double quote, and want to capture the part between these double quotes. This is spelled as 'cfg "(.?)"'. Since you don't want to deal with escape chars, the best way is to use a raw single quoted string:

exp = r'cfg *"(.+?)"'

now since you're going to reuse this expression in a loop, you might as well compile it already:

exp = re.compile(r'cfg *"(.+?)"')

so now exp is a re.pattern object instead of string. To use it, you call it's search(<text>) method, with your current line as argument. If the line matches the expression, you'll get a re.match object, else you'll get None:

>>> match = exp.search('foo bar "baaz" boo')
>>> match is None
True
>>> match = exp.search('foo bar -cfg "RGB 888; MODE 3" tagada "tsoin"')
>>> match is None
False
>>> 

To get the part between the double quotes, you call match.group(1) (second captured group, the first one being the one matchin the whole expression)

>>> match.group(0)
'cfg "RGB 888; MODE 3"'
>>> match.group(1)
'RGB 888; MODE 3'
>>> 

Now you just have to learn and make correct use of files... First hint: files are context managers that know how to close themselves. Second hint: files are iterable, no need to read the whole file in memory. Third hint : file.write("text") WONT append a newline after "text".

If we glue all this together, your code should look something like:

import re
search_term = re.compile(r'cfg *"(.+?)"')

with open("testlist.csv", "r") as ifile:
    with open("result.txt", "w") as ofile:
        for line in ifile:
            match = search_term.search(line)
            if match:
                ofile.write(match.group(1) + "\n")