Sed command in python

Question

My input is as

Type combinational  function (A B)

Want output to be

Type combinational 
function (A B)

I used code and its working

sed 's/\([^ ]* [^ ]*\) \(function.*\)/\1\n\2/' Input_file

When I use this code inside python script using os.system and subprocess its giving me error. How can I execute this sed inside python script. Or how can I write python code for above sed code. Python code used

cmd='''
sed 's/\([^ ]* [^ ]*\) \(function.*\)/\1\n\2/' Input_file
'''
subprocess.check_output(cmd, shell=True)

Error is

sed: -e expression #1, char 34: unterminated `s' command

Try `r` before the opening `'''`. – bereal Nov 05 '20 at 11:17 — bereal, Nov 05 '20 at 11:17

tripleee · Accepted Answer · 2020-11-05T11:27:20.250

The \n in the string is being substituted by Python into a literal newline. As suggested by @bereal in a comment, you can avoid that by using r'''...''' instead of '''...''' around the script; but a much better solution is to avoid doing in sed what Python already does very well all by itself.

with open('Input_file') as inputfile:
   lines = inputfile.read()
lines = lines.replace(' function', '\nfunction')

This is slightly less strict than your current sed script, in that it doesn't require exactly two space-separated tokens before the function marker. If you want to be strict, try re.sub() instead.

import re
# ...
lines = re.sub(r'^(\S+\s+\S+)\s+(function)', r'\1\n\2', lines, re.M)

(Tangentially, you also want to avoid the unnecessary shell=True; perhaps see Actual meaning of 'shell=True' in subprocess)

score 0 · Answer 2 · edited Nov 15 '21 at 01:26

Although the solutions 1 and 2 are the shortest valid way to get your code running (on Unix), i'd like to add some remarks:

a. os.system() has some issues related to it, and should be replaced by subprocess.call("your command line", shell=False). Regardless of using os.system or subprocess.call, shell=True implies a security risk.

b. Since sed (and awk) are tools that rely heavily on regular expressions it is recommended, when building python for maintainability, to use native python code. In this case use the re, regular expression module, which has a regexp optimized implementation.

Sed command in python

2 Answers2