Read file and write blocks of lines to new files using loop

Question

I'm fairly new to python and I am trying to write a script which reads x lines from a file and then writes that to new file. I would like it to iterate over all the subsequent lines writing to separate new files each time. The first 7 lines need to go in every new file.

Example format of the file to read is below:

Line 1: Data for EVERY new file
...
...
Line 8: Data to write to file one
...
...
Line 49: Data to write to file two
...
...

I have tried using itertools.islice but am not sure how to loop it for subsequent lines as well as into new files each time.

Sample code:

    import os
from itertools import islice

#os.chdir("directory where XDATCAR is")
#ignore if .py is in the same directory


with open('XDATCAR' , 'r') as f:
    for line in islice(f, 2, 7,):
        print(line, end='')
with open('POSCAR','w') as n:
            n.writelines()

I get the error:

ValueError: I/O operation on closed file.

Which I understand to be because 'with' closes the file after execution.

What I would like as output for each new file:

File 1:
First 7 lines of read file
lines 8 to 48

File 2:
First 7 lines of read file
lines 49 to 89
...
etc

As an aside, you basically never need to `os.chdir()` to where a file is. See [What exactly is current working directory?](https://stackoverflow.com/questions/45591428/what-exactly-is-current-working-directory) — tripleee, Feb 01 '22 at 06:34

score 0 · Answer 1 · answered Jan 31 '22 at 13:52

I did this by saving the contents of "XDATCAR" into a list and iterating over it.

# read the lines
f=open("XDATCAR")
lines=f.readlines()
f.close()

file_number=0 # counter for the files
line_counter=0 # keeps track of the number of lines written in the file so far

for i in range(8,len(lines)):
    if line_counter==0: # we started writing to a file
        f=open("file_"+str(file_number)+".txt","w") # open file
        for j in range(7):# write the first seven lines
            f.write(lines[j])

    f.write(lines[i])# write one line
    line_counter+=1 # increment line counter
    
    if line_counter == 40:
        f.close()
        file_number+=1 # start writing to a new file
        line_counter=0 

if f.closed == False: # the file was not closed properly in the for loop
    f.close()

For some reason it skips the 8th line, 48th etc? Other than that it works perfectly. I just need to put each file in it's own directory now.. Thanks! — Kit Barker, Jan 31 '22 at 14:32

xtofl · Answer 2 · 2022-02-01T06:14:39.973

This sounds like a job for more_itertools.chunked:

This code

reads in the header lines
uses the ichunked generator to make large chunks out of the rest
uses the chain generator to join header and chunk lines

import itertools
import more_itertools

def write_chunk(i, chunk_lines):
    output_filename = f"file_{i}.txt"
    with open(output_filename, "w") as w:
        w.writelines(chunk_lines)

with open("XATCAR") as lines:
    header = tuple(itertools.islice(lines, 0, 7))
    chunks = more_itertools.chunked(lines, 42)

    for i, chunk in enumerate(chunks):
        write_chunk(i, itertools.chain(header, chunk))

score 0 · Answer 3 · answered Jan 31 '22 at 14:30

Let us write down what you want to do:

read 7 lines from the input file. Those lines should be saved because they will be used for all output file
read 41 lines from the input file, and write them to the output file after the 7 initial lines
iterate step 2 until the input file is exhausted

Code could be:

...
with open('XDATCAR' , 'r') as f:
    # read 7 lines into a list:
    header = [next(f) for _ in range(7)]
    filenum = 0
    while True:
        # output file
        with open(f'POSTCAR{filenum}', 'w') as out:
            for line in header:   # copy 7 initial lines
                out.write(line)
            for i in range(41): # try to copy 41 lines
                try:
                    out.write(next(f))
                except StopIteration:
                    break
        filenum += 1
    # remove last file if no line was used
    if i == 0:
        os.remove(f'POSTCAR{filenum}')

Read file and write blocks of lines to new files using loop

3 Answers3