Unix cat function (cat * > merged.txt) in Python?

Question

Is there a way to use the cat function from Unix in Python or something similar once a directory has been established ? I want to merge files_1-3 together into merged.txt

I would usually just find the directory in Unix and then run

cat * > merged.txt 

file_1.txt
file_2.txt
file_3.txt

merged.txt

Also related: http://stackoverflow.com/questions/1001538/how-do-i-concatenate-files-in-python — qehgt, Nov 27 '13 at 22:41

Ashwini Chaudhary · Answer 1 · 2013-11-27T21:33:48.647

4

Use the fileinput module:

import fileinput
import glob
with open('/path/to/merged.txt', 'w') as f:
    for line in fileinput.input(glob.glob('/path/to/files/*')):
        f.write(line)
    fileinput.close()

edited Nov 27 '13 at 21:33

answered Nov 27 '13 at 21:24

Ashwini Chaudhary

244,495
58
464
504

how does it know which directory it is in ? – O.rka Nov 27 '13 at 21:25
@draconisthe0ry You can pass the directory path to `glob.glob`. – Ashwini Chaudhary Nov 27 '13 at 21:25
IIRC, `fileinput` objects should be closed just like you close a regular file. It wouldn't hurt to `fileinput.close()` after that for loop I think... – mgilson Nov 27 '13 at 21:29
works, but I need a space between `content_file_1` and `content_file_2` , otherwise at least two words will merge from both files. How can I do that? – Just Me Aug 29 '22 at 19:46

score 2 · Accepted Answer · edited May 23 '17 at 10:25

2

As we know we are going to use "Unix" cat command (unless you are looking for a pythonic way or being performance concious)

You can use

import os
os.system("cd mydir;cat * > merged.txt")

or

as pointed by 1_CR (Thanks) and explained here Python: How to Redirect Output with Subprocess?

edited May 23 '17 at 10:25

Community

1
1

answered Nov 27 '13 at 22:13

user2390183

975
8
17

you may have a point however [check this](http://docs.python.org/2/library/subprocess.html#replacing-older-functions-with-the-subprocess-module) – iruvar Nov 27 '13 at 22:18
There are a score of reasons not to just shell out; among them, the overhead of spawning a new process to spawn another new process just to read files and write them out; secondly: security, if you're using user input to determine where to read or write. Also, shelling out adds a selection of issues, like ensuring cwd is correct, $PATH, etc., and is non-portable. – Dan Udey Nov 27 '13 at 22:50
1

Valid points. however, :) Within the context of problem raised by OP- "Is there a way to use the cat function from Unix in Python", I guess OP wants to execute simple task automation than building real application. And I still feel this is one of the moments for which you ( say you never used, I will accept your Vote down :)) and me used "os.system" calls to execute such simple tasks by being practical. – user2390183 Nov 27 '13 at 23:13

score 0 · Answer 3 · answered Nov 27 '13 at 21:27

Use fileinput. Say you have a python file merge.py with the following code, you could call it like so merge.py dir/*.txt. File merged.txt gets written to current dir. By default, fileinput iterates over the list of files passed on the command line, so you can let the shell handle globbing

#!/usr/bin/env python
import fileinput
with open('merged.txt', 'w') as f:
    for line in fileinput.input():
        f.write(line)

Unix cat function (cat * > merged.txt) in Python?

3 Answers3

Linked

Related