Pick parts from a txt file and copy to another file with python

Question

I'm in trouble here. I need to read a file. Txt file that contains a sequence of records, check the records that I want to copy them to a new file. The file content is like this (this is just an example, the original file has more than 30 000 lines):

AAAAA|12|120 #begin file
00000|46|150 #begin register
03000|TO|460 
99999|35|436 #end register
00000|46|316 #begin register
03000|SP|467
99999|33|130 #end register
00000|46|778 #begin register
03000|TO|478
99999|33|457 #end register
ZZZZZ|15|111 #end file

The records that begin with 03000 and have the characters 'TO' must be written to a new file. Based on the example, the file should look like this:

AAAAA|12|120 #begin file
00000|46|150 #begin register
03000|TO|460 
99999|35|436 #end register
00000|46|778 #begin register
03000|TO|478
99999|33|457 #end register
ZZZZZ|15|111 #end file

Code:

file = open("file.txt",'r')
newFile = open("newFile.txt","w")    
content = file.read()
file.close()
# here I need to check if the record exists 03000 characters 'TO', if it exists, copy the recordset 00000-99999 for the new file.

I did multiple searches and found nothing to help me. Thank you!

What do you think of my solution, please ? – eyquem May 19 '13 at 15:06 — eyquem, May 19 '13 at 15:06

elyase · Answer 1 · 2013-05-17T15:00:17.790

6

with open("file.txt",'r') as inFile, open("newFile.txt","w") as outFile:
    outFile.writelines(line for line in inFile 
                       if line.startswith("03000") and "TO" in line)

If you need the previous and the next line, then you have to iterate inFile in triads. First define:

def gen_triad(lines, prev=None):
    after = current = next(lines)
    for after in lines:
        yield prev, current, after
        prev, current = current, after

And then do like before:

outFile.writelines(''.join(triad) for triad in gen_triad(inFile) 
                   if triad[1].startswith("03000") and "TO" in triad[1])

edited May 17 '13 at 15:00

answered May 15 '13 at 21:00

elyase

39,479
12
112
119

1

note that he wants to write the first and the last line. and also the previous and the next line of each match.. read carefully before answering.. – IcyFlame May 16 '13 at 15:36

score 0 · Answer 2 · answered May 17 '13 at 03:00

import re

pat = ('^00000\|\d+\|\d+.*\n'
       '^03000\|TO\|\d+.*\n'
       '^99999\|\d+\|\d+.*\n'
       '|'
       '^AAAAA\|\d+\|\d+.*\n'
       '|'
       '^ZZZZZ\|\d+\|\d+.*')
rag = re.compile(pat,re.MULTILINE)

with open('fifi.txt','r') as f,\
     open('newfifi.txt','w') as g:
    g.write(''.join(rag.findall(f.read())))

For files with additional lines between lines beginning with 00000, 03000 and 99999, I didn't find simpler code than this one:

import re

pat = ('(^00000\|\d+\|\d+.*\n'
       '(?:.*\n)+?'
       '^99999\|\d+\|\d+.*\n)'
       '|'
       '(^AAAAA\|\d+\|\d+.*\n'
       '|'
       '^ZZZZZ\|\d+\|\d+.*)')
rag = re.compile(pat,re.MULTILINE)

pit = ('^00000\|.+?^03000\|TO\|\d+.+?^99999\|')
rig = re.compile(pit,re.DOTALL|re.MULTILINE)

def yi(text):
    for g1,g2 in rag.findall(text):
        if g2:
            yield g2
        elif rig.match(g1):
            yield g1

with open('fifi.txt','r') as f,\
     open('newfifi.txt','w') as g:
    g.write(''.join(yi(f.read())))

score -1 · Answer 3 · answered May 15 '13 at 20:50

-1

file = open("file.txt",'r')
newFile = open("newFile.txt","w")    
content = file.readlines()
file.close()
newFile.writelines(filter(lambda x:x.startswith("03000") and "TO" in x,content))

answered May 15 '13 at 20:50

Joran Beasley

110,522
12
160
179

if you use `readlines` it should only pick up one line at a time. if you use `read` it will get everything (or up to X bytes) – Joran Beasley May 16 '13 at 18:50
Very good Joran Beasley, but I would not just pick up the contents of a line, but a set of lines as the example. I tried changing the code you posted, but I could not because it had not yet worked with lambda functions. Thank you! – fredsilva May 16 '13 at 18:51
see my comment i posted when you put this earlier :) – Joran Beasley May 16 '13 at 18:51

score -1 · Answer 4 · answered May 16 '13 at 14:25

This seems to work. The other answers seem to only be writing out records that contain '03000|TO|' but you have to write out the record before and after that as well.

    import sys
# ---------------------------------------------------------------
# ---------------------------------------------------------------
# import file
file_name = sys.argv[1]
file_path = 'C:\\DATA_SAVE\\pick_parts\\' + file_name
file = open(file_path,"r")
# ---------------------------------------------------------------
# create output files
output_file_path = 'C:\\DATA_SAVE\\pick_parts\\' + file_name + '.out'
output_file = open(output_file_path,"w")
# create output files

# ---------------------------------------------------------------
# process file

temp = ''
temp_out = ''
good_write = False
bad_write = False
for line in file:
    if line[:5] == 'AAAAA':
        temp_out += line 
    elif line[:5] == 'ZZZZZ':
        temp_out += line
    elif good_write:
        temp += line
        temp_out += temp
        temp = ''
        good_write = False
    elif bad_write:
        bad_write = False
        temp = ''
    elif line[:5] == '03000':
        if line[6:8] != 'TO':
            temp = ''
            bad_write = True
        else:
            good_write = True
            temp += line
            temp_out += temp 
            temp = ''
    else:
        temp += line

output_file.write(temp_out)
output_file.close()
file.close()

Output:

AAAAA|12|120 #begin file
00000|46|150 #begin register
03000|TO|460 
99999|35|436 #end register
00000|46|778 #begin register
03000|TO|478
99999|33|457 #end register
ZZZZZ|15|111 #end file

score -1 · Answer 5 · answered May 18 '13 at 19:37

-1

Does it have to be python? These shell commands would do the same thing in a pinch.

head -1 inputfile.txt > outputfile.txt
grep -C 1 "03000|TO" inputfile.txt >> outputfile.txt
tail -1 inputfile.txt >> outputfile.txt

answered May 18 '13 at 19:37

Anand

7,654
9
46
60

score -2 · Answer 6 · answered May 15 '13 at 22:56

-2

# Whenever I have to parse text files I prefer to use regular expressions
# You can also customize the matching criteria if you want to
import re
what_is_being_searched = re.compile("^03000.*TO")

# don't use "file" as a variable name since it is (was?) a builtin 
# function 
with open("file.txt", "r") as source_file, open("newFile.txt", "w") as destination_file:
    for this_line in source_file:
        if what_is_being_searched.match(this_line):
            destination_file.write(this_line)

and for those who prefer a more compact representation:

import re

with open("file.txt", "r") as source_file, open("newFile.txt", "w") as destination_file:
    destination_file.writelines(this_line for this_line in source_file 
                                if re.match("^03000.*TO", this_line))

answered May 15 '13 at 22:56

EvensF

1,479
1
10
17

note that he wants to write the first and the last line. and also the previous and the next line of each match.. read carefully before answering.. – IcyFlame May 16 '13 at 15:34
Well, he said "The records that begin with 03000 and have the characters 'TO' must be written to a new file." If it's not what he is looking for he should be more clear about what he wants... – EvensF May 16 '13 at 19:00
I hope that you red the whole question before answering. Can't you see the output he expects?? – IcyFlame May 19 '13 at 08:13

score -2 · Answer 7 · answered May 16 '13 at 15:28

code:

fileName = '1'

fil = open(fileName,'r')

import string

##step 1: parse the file.

parsedFile = []

for i in fil:

    ##tuple1 = (1,2,3)    

    firstPipe = i.find('|')

    secondPipe = i.find('|',firstPipe+1)

    tuple1 = (i[:firstPipe],\
                i[firstPipe+1:secondPipe],\
                 i[secondPipe+1:i.find('\n')])

    parsedFile.append(tuple1)


fil.close()

##search criterias:

searchFirst = '03000'  
searchString = 'TO'  ##can be changed if and when required

##step 2: used the parsed contents to write the new file

filout = open('newFile','w')

stringToWrite = parsedFile[0][0] + '|' + parsedFile[0][1] + '|' + parsedFile[0][2] + '\n'

filout.write(stringToWrite)  ##to write the first entry

for i in range(1,len(parsedFile)):

    if parsedFile[i][1] == searchString and parsedFile[i][0] == searchFirst:

        for j in range(-1,2,1):

            stringToWrite = parsedFile[i+j][0] + '|' + parsedFile[i+j][1] + '|' + parsedFile[i+j][2] + '\n'

            filout.write(stringToWrite)


stringToWrite = parsedFile[-1][0] + '|' + parsedFile[-1][1] + '|' + parsedFile[-1][2] + '\n'

filout.write(stringToWrite)  ##to write the first entry

filout.close()

I know that this solution may be a bit long. But it is quite easy to understand. And it seems an intuitive way to do it. And I have already checked this with the Data that you have provided and it works perfectly.

Please tell me if you need some more explanation on the code. I will definitely add the same.

Well I did not think the wrong answer would get as many upvotes.... when my answer is right and did not get an up vote!!! In fact I get a down vote. Great way to run a forum!!! #sarcastic — IcyFlame, May 19 '13 at 08:14
@IcyFlame, I dont know why you got downvoted(BTW I have also received 3 downvotes) but why do you think my answer is wrong? — elyase, May 20 '13 at 16:33

fredsilva · Answer 8 · 2013-05-16T20:03:29.827

-2

I tip (Beasley and Joran elyase) very interesting, but it only allows to get the contents of the line 03000. I would like to get the contents of the lines 00000 to line 99999. I even managed to do here, but I am not satisfied, I wanted to make a more cleaner. See how I did:

    file = open(url,'r')
    newFile = open("newFile.txt",'w')
    lines = file.readlines()        
    file.close()
    i = 0
    lineTemp = []
    for line in lines:                     
        lineTemp.append(line)                       
        if line[0:5] == '03000':
            state = line[21:23]                                
        if line[0:5] == '99999':
            if state == 'TO':
                newFile.writelines(lineTemp)                    
            else:
                linhaTemp = []                                                                            
        i = i+1                      
    newFile.close()

Suggestions... Thanks to all!

edited May 16 '13 at 20:03

answered May 16 '13 at 19:08

fredsilva

257
1
3
12

I could modify my answer to include the previous and the next line if needed. I see in your code you don't include them, do you also need the lines starting with '99999'? – elyase May 16 '13 at 22:40
@elyase don't just say you could modify it. Just do it as the person who asked the question needs it! – IcyFlame May 19 '13 at 08:15
Well, I actually modified it, I am just not sure what he wants as his code contradicts his example. – elyase May 19 '13 at 11:16

Pick parts from a txt file and copy to another file with python

8 Answers8

Linked