How to use python to reformat a file

Question

I need a python script to reformat a poorly formatted .txt file. I was not thinking in advanced while doing my code and ended up with a very poorly formatted .txt file. Below is an excerpt from my file(the file is 3000 lines in the full file):

Below is what I want the file to look like. Basically I want python to go through each line and change from the old bad format to this new format. Just as a heads up there are thousands of lines:

temp,cur_alt,zero_alt,dif
27.80,92.51,76.02,16.49
27.80,91.49,76.02,15.47
27.80,90.99,76.02,14.97

You could read the lines into a list and each time you encounter an empty line start a new list. So you end up with a list of lists. Then use the [csv](https://docs.python.org/3/library/csv.html) module to create the required format. — Paul Rooney, Oct 07 '19 at 23:08
If you search in your browser for Python file and string processing, you'll find references that can explain this much better than we can manage here. IN particular, look at the `join` method for strings. — Prune, Oct 07 '19 at 23:21
You may want to take a look at iterating over a list in chunks, taking advantage of the fact that you can iterate over all lines of a file: https://stackoverflow.com/questions/434287/what-is-the-most-pythonic-way-to-iterate-over-a-list-in-chunks — 10762409, Oct 07 '19 at 23:34
An awk: `$ awk -v RS= 'BEGIN{print "temp,cur_alt,zero_alt,dif"} {gsub(/\n/,","); print}' your_file` — dawg, Oct 08 '19 at 00:02

neutrino_logic · Accepted Answer · 2019-10-08T00:13:25.510

This question was rather severely downvoted but it's kind of an interesting problem. I've been messing about with generators and regex and thought this would be good practice, so:

import re

def readfile(target):
    """use regex to capture and yield only lines that match float"""
    regex = re.compile(r'([0-9]+|).[0-9]+')
    with open(target, 'r') as fin:
        for line in fin:
            line = line.rstrip()
            if regex.search(line):
                yield line

def writefile(target, destination):
    """accumulates generator output to str, writes line-by-line to new file"""
    datum = readfile(target)
    with open(destination, 'w') as fout:
        fout.write("temp,cur_alt,zero_alt,dif\n")
        temp = []
        count = 0
        while True:
            try: 
                test = next(datum)
                temp.append(test)
                count += 1
            except StopIteration:
                return
            if count % 4 == 0:
                result = ','.join(temp)
                temp = []
                fout.write(result + '\n')

if __name__ == "__main__":
    target = input("Enter full source file name: ")
    destination = input("Enter full destination file name: ")
    writefile(target, destination)

Seems to work. Won't overwrite your raw data at least.

How to use python to reformat a file

1 Answers1