Python (or Python+Perl) : set value in a configuration file

Question

I can't find a question with an answer that work as i want.

I use to script in bash, but I'm noticing that bash is way too inefficient when it comes to large operations so i'm rewriting my code in python.

But i'm noob in python! So i google a lot but here i have a problem... there are a lot of sed operations in my bash script and i would like to know whats the way to recreate this (and this will be the starting point to rewrite all my sed operations) in python:

sed -i -e "s/\(log_min_messages = \).*/\1notice/" "/opt/PostgreSQL/9.6/data/postgresql.conf"

Which substitute everything after "log_min_messages" in that file with "notice".

Basically that open postgresql.conf file, search for matching pattern "log_min_messages = *" (where * indicates that it doesnt matter what is after '=' ) and replace '*' with the string 'notice'.

Example:

before

log_min_messages = something

after that sed command:

log_min_messages = notice

I wonder if i have to use 're' ?

EDIT: An idea is to implement Perl commands in the python script, so if anyone knows how to do that in perl would be a wonderful answer anyway

tought about it, unluckily this script has many things to do...not only "sedding" around...by the way I could call some perl commands from python through the shell...in this case what would it be? (editing question too) — Wyatt Gillette, Nov 24 '17 at 14:27
Maybe some `pythoners` are not familiar with `bash`, so you should explain in plain, and give some example or so. — Kinght 金, Nov 24 '17 at 14:41

Rong Zhao · Answer 1 · 2017-11-25T13:06:12.617

ConfigParser can read/write ini configure file.

I am not sure it can meet your requirement or not.

refer to: https://docs.python.org/2/library/configparser.html

the first step is read configurations from file, then change the value in config value, at last, write the configure back into file.

Hm... sed is more simple.. :)

Put a solution based on Alex Martelli's Solution

if we have test2.cfg like this:

key1=abcd
key2=abcd3

Then use code below:

import ConfigParser


class FakeHeader(object):

    def __init__(self, fp, section='zconfig'):
        self.section = '[%s]\n' % section
        self.fp = fp

    def readline(self):
        if self.section:
            try:
                return self.section
            finally:
                self.section = None
        else:
            return self.fp.readline()


class ZConfigParser(ConfigParser.ConfigParser, object):

    def __init__(self, default_section='zconfig'):
        ConfigParser.ConfigParser.__init__(self)
        self.default_section = default_section

    def read(self, fp):
        new_fp = FakeHeader(open(fp), self.default_section)
        return super(ZConfigParser, self).readfp(new_fp)

    def write(self, fp):
        fp = open(fp, "w")
        for (key, value) in self.items(self.default_section):
            fp.write("%s = %s\n" % (key, str(value).replace('\n', '\n\t')))
        fp.write("\n")

    def get(self, key):
        return super(ZConfigParser, self).get(self.default_section, key)

    def set(self, key, value):
        super(ZConfigParser, self).set(self.default_section, key, value)

# here you can use it to read/write a configure file without section in somehow graceful method. 

zconfig = ZConfigParser()

# read file
zconfig.read("test2.cfg")

print zconfig.get('key1')

zconfig.set('key1', 'alfasjdlfabalalla')

# then you can check test3.cfg which key1=alfasjdlfabalalla
zconfig.write('test3.cfg')

Hope it can help you.

that would be great if only postgresql.conf had sections :( in fact, there aren't...one could add a section at the beginning and then remove it via some simple commands tought...i'll give a try but i think it's really only for "standard" conf files... — Wyatt Gillette, Nov 24 '17 at 14:50
@WyattGillette Sir, that is no problem, please refer this link https://stackoverflow.com/questions/2885190/using-pythons-configparser-to-read-a-file-without-section-name I used same method to read such files. I think it is easy to improve this to read/write configure file without section. — Rong Zhao, Nov 25 '17 at 02:06

cdarke · Answer 2 · 2017-11-26T09:56:08.457

The conversion from sed is quite straightforward, we have to create a temporary file (outfname) to write the new text to, then rename it. Python regular expressions are extended (as is Perl), so the parentheses are not escaped.

The leading r on a string r"..." is a raw string, which means it won't try to translate escaped characters like \1.

import re
import os

fname = "/opt/PostgreSQL/9.6/data/postgresql.conf"
outfname = fname + '.new'

with open(fname, 'r') as infile, open(outfname, 'w') as outfile:
    for line in infile:
        line = re.sub(r"(log_min_messages = ).*", r"\1notice", line)
        outfile.write(line)

os.rename(outfname, fname)

However, you don't need regular expressions for this. So you could omit the import re and replace the line = re.sub(....) line with:

        if 'log_min_messages = ' in line:
            line = line.split('=')[0] + ' notice\n'

The in operator tests for the presence of the sub-string, you might be able to use line.startswith() instead, but I'm not certain of your file format. Note that we have to add the \n newline here, by default the re methods (and sed) do not match .* with a newline.

In general, if you can get the job using string methods rather than a regular expression then the code is simpler and more efficient (though its doubtful in this case if you will see a difference).

Python (or Python+Perl) : set value in a configuration file

2 Answers2