4

I've searched and found code below that can interleave 2 files line by line into new one.

from itertools import izip_longest
from contextlib import nested

with nested(open('foo'), open('bar')) as (foo, bar):
    for line in (line for pair in izip_longest(foo, bar)
                      for line in pair if line):
        print line.strip()

I have multiple files and like to have 2 or more consecutive lines to interleave one after another. I like to be able to choose number of lines depend on the job. Total number of lines in each file may not be the same, but the pattern for number of lines for each element on all files are always the same. How can I achieve my goal?

input:
fileA
lineA1
lineA2
lineA3
......

fileB
lineB1
lineB2
lineB3
......

For 2 lines output:
lineA1
lineA2
lineB1
lineB2
.....

For 3 lines output:
lineA1
lineA2
lineA3
lineB1
lineB2
lineB3
....

Thank you.

@xealits Thanks a million. Your codes work like a champ. Have a nice day!

mlov
  • 55
  • 3

1 Answers1

1

This should work:

from itertools import islice

# number of sequential lines to read from each file
N = 2
# files that are read
files = [open(n) for n in ['foo', 'bar', 'baz']]

line = ''.join([''.join(islice(f, N)) for f in files])[:-1]
while line:
    print(line)
    line = ''.join([''.join(islice(f, N)) for f in files])[:-1]

[f.close() for f in files]

-- here the files are opened and closed manually in list comprehensions instead of using with and nested or ExitStack; when islice called on a file reads first N lines from it (and when the lines are read they are not in the file object any more -- they pop out of it) (also, one can loop through a file with [line for line in file]); ''.join joins items of the given list/tuple/iterable object into a string with '' as the separator; since print adds newline to the printed string, [:-1] is to remove the last character of the resulting string from files, which is newline character.

If you read only 2 files, then with is fine:

from itertools import islice

# number of sequential lines to read from each file
N = 2

with open('foo') as foo, open('bar') as bar:
    line = ''.join(islice(foo, N)) + ''.join(islice(bar, N))[:-1]
    while line:
        print(line)
        line = ''.join(islice(foo, N)) + ''.join(islice(bar, N))[:-1]

Some references for the used stuff:

How to read file N lines at a time in Python?

nested is deprecated

Python 3 uses ExitStack for nested features in with

ExitStack

Community
  • 1
  • 1
xealits
  • 4,224
  • 4
  • 27
  • 36
  • Closing the files is **always** a good idea. Even if they are closed on your current version of your specific python interpreter, there is absolutely no guarantee they will be in next release. And in fact, neither Jython nor Pypy close files automatically in this case. – spectras Jul 09 '16 at 01:11
  • @spectras, yep. But at the same time `nested` is deprecated. And apparently one [has to use StackExit in Python 3](http://stackoverflow.com/questions/4617034/how-can-i-open-multiple-files-using-with-open-in-python#comment22683982_4617069).. So, if the target is any python interpreter (Jython/Pypy), maybe one needs to make the list of the files, loop through them and close them manualy at the end. – xealits Jul 09 '16 at 01:17
  • Indeed. Though here, as you're reading all files before looping through the line, you could just open them one after the other, there is no need for nested or stackexit. – spectras Jul 09 '16 at 01:20
  • 1
    To xealits, please change your code to interleave 2 or more consecutive lines from each file. Just edited my post for clarification. Thank for your help. – mlov Jul 09 '16 at 03:10
  • @xealits Thanks a million. Your codes work like a champ. Have a nice day! – mlov Jul 09 '16 at 17:44
  • @xealits, Sorry, just accepted your answer. Again, Thanks. – mlov Jul 10 '16 at 00:15