Remove lines from text file that don't meet condition

Question

I have a text file that contains the following:

======== Data: 00:05:08.627012 =========

1900-01-01 00:05:08.627012 ; 0 ; 1.16198 ; 10000000.0

1900-01-01 00:05:08.627012 ; 1 ; 1.16232 ; 10000000.0

========= Data: 00:05:12.721536 =========

1900-01-01 00:05:08.627012 ; 0 ; 1.16198 ; 10000000.0

1900-01-01 00:05:12.721536 ; 0 ; 1.16209 ; 1000000.0

1900-01-01 00:05:08.627012 ; 1 ; 1.16232 ; 10000000.0

I am trying to convert it to a csv where each item with a semicolon after it goes into its own cell. Here is an idea of the desired result.

I don't want to include the lines that have the = signs in the text file. I currently am using the following code:

txt_file = open('Data/Mkt_data_test.txt', 'r')
lines = txt_file.readlines()
txt_file.close()

header_line = ['Time,', 'Bid/Ask,', 'Price,', 'Volume,']

data_lines = []

for line in lines:
    if '=' not in line:
        time_data = line.split('\n')
        for time in time_data:
            data_lines.append(time+'\n')
            data_lines = [data.replace(';', ',') for data in data_lines]

finished_file = open('mktDataFormat.csv', 'w')
finished_file.writelines(header_line)
finished_file.writelines(data_lines)
finished_file.close()

This correctly writes the lines that don't contain an equal sign but there are blank rows where the lines with an '=' are and where there is also just an empty row in the text file.

How can I get rid of those blank lines?

Possible duplicate of [using Python for deleting a specific line in a file](https://stackoverflow.com/questions/4710067/using-python-for-deleting-a-specific-line-in-a-file) — Torxed, Feb 28 '19 at 20:24
Why are you splitting on new lines if you already called readlines — Mitch, Feb 28 '19 at 20:28

score 0 · Answer 1 · answered Feb 28 '19 at 20:30

for line in lines:
    if '=' not in line:
        time_data = line.split('\n')
        for time in time_data:
            data_lines.append(time+'\n')
        data_lines = [data.replace(';', ',') for data in data_lines]

Try that and let me know

score 0 · Accepted Answer · answered Feb 28 '19 at 20:39

Your problem is that your program did not skip blank lines, and thus was treating blank lines as data. I added a check (and kind of reworked your code a bit), to ensure there are no blank lines.

txt_file = open('Data/Mkt_data_test.txt', 'r')
lines = txt_file.readlines()
txt_file.close()

header_line = ['Time,', 'Bid/Ask,', 'Price,', 'Volume,\n']

data_lines = []

for line in lines:
    if '=' not in line and line.strip() != "":
        line = line.replace(';', ',')
        data_lines.append(line)

 finished_file = open('mktDataFormat.csv', 'w')
 finished_file.writelines(header_line)
 finished_file.writelines(data_lines)
 finished_file.close()

Remove lines from text file that don't meet condition

2 Answers2