join rows of different files in python

Question

If text1

text2

and text3

What is the most direct way to join the rows of the different files together and write the resulting text to a file?

1 2 3 4 5 6 7 8 9 10 11
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7

I tried looping through the files and use file.read() but this merges the files vertically

It can be done without pandas too. You should have at least tried something. — roganjosh, Apr 20 '18 at 19:33
@roganjosh: Yes. I tried looping through the files and use file.read() and tried other options (https://stackoverflow.com/questions/13613336/python-concatenate-text-files), but they all merge files vertically — HappyPy, Apr 20 '18 at 19:39
No need for advanced tools. Just a unix commend: `paste -d' ' text1 text2 text3` — llllllllll, Apr 20 '18 at 19:42
@liliscent: Thanks! But can I use that in a .py script? Or do you mean doing it via the command line? — HappyPy, Apr 20 '18 at 19:53
@HappyPy It's for commend line. You can do it in Python as well. `result = Popen(('paste', '-d ', 'text1', 'text2', 'text3'))`. But if you're already writing Python script, I recommend you write pure Python. — llllllllll, Apr 20 '18 at 19:59

score 2 · Answer 1 · answered Apr 20 '18 at 19:38

I think pandas is easier, but you can do it without pandas.

file_names = ['file1.txt','file2.txt','file3.txt']
res = []
for name in file_names:
    with open(name,'rb') as f:
        res.append(f.readlines())

res = [(' '.join(map(lambda row:row.strip(),line)) + '\n') for line in zip(*res)]

output:

['1 2 3 4 5 6 7 8 9 10 11\n',
 '1 2 3 4 5 6 7 8 9\n',
 '1 2 3 4 5 6 7 8\n',
 '1 2 3 4 5 6 7\n']

chickity china chinese chicken · Accepted Answer · 2018-04-20T20:18:09.437

2

Another option is use zip to merge the lines in the text files:

with open('text1', 'r') as f1:
    t1 = f1.readlines()

with open('text2', 'r') as f2:
    t2 = f2.readlines()

with open('text3', 'r') as f3:
    t3 = f3.readlines()

for item in zip(t1,t2,t3):
    print ' '.join([line.strip() for line in item])

result:

1 2 3 4 5 6 7 8 9 10 11
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7

Edit: More generalized solution:

files = ['text1', 'text2', 'text3']

contents = []

for f in files:
    with open(f, 'r') as fn:
        contents.append(fn.readlines())

for entry in zip(contents):
    for item in entry:
        print ' '.join([line.strip() for line in item])

edited Apr 20 '18 at 20:18

answered Apr 20 '18 at 19:45

chickity china chinese chicken

7,709
2
20
49

Thanks. Is there a way to make this more general, like defining a list of files at the beginning, instead of having to write "with open..." for each file? – HappyPy Apr 20 '18 at 19:53
Surely there is a way, but I can't think of one this moment. If I figure one out, I'll post it. – chickity china chinese chicken Apr 20 '18 at 20:05
@HappyPy see edit in updated answer, it's not very clean or optimized, but it seems to work. – chickity china chinese chicken Apr 20 '18 at 20:19
1

@davedwards Depending on how many files OP is working with (and/or the size of them), using `readlines` on each one might cause the script to run out of memory though. Looks like the same potential issue in the other answer as well. – G_M Apr 20 '18 at 20:30
1

@DeliriousLettuce True, especially if it were to become an issue for others using this approach in future – chickity china chinese chicken Apr 20 '18 at 20:36

join rows of different files in python

2 Answers2