Open Multiple Files, Helper Function

Question

I am aware of other solutions to this problem, such as using with open as, but first I want to understand why my code is, or is not, a good solution.

I am trying to open two CSV files, one for reading and one for writing. The script should continue only if both files were successfully opened. My code seems to accomplish that but I would like to know a couple things:

What's the most Pythonic way to accomplish this, and why?
Is it bad practice to exit a script from within a helper function?

Original Code:

input_file = 'in_file.csv'
output_file = 'out_file.csv'


def open_file(file, mode):

    try:
        fp = open(file, mode)
    except IOError as e:
        print "Error: cannot open {0}".format(file)
        if e.errno == errno.EACCES:
            print "\tPermission denied."
            print "\tError message: {0}".format(e)
            sys.exit()
        # Not a permission error.
        print "\tDoes file exist?"
        print "\tError message: {0}".format(e)
        sys.exit()
    else:
        return fp


def main():

    # Open files in binary read/write mode for platform independence.
    out_csv = open_file(output_file, 'wb')
    in_csv = open_file(input_file, 'rb')

    # Do stuff with the files
    #
    # with out_csv:
    #
    #   writer = csv.writer(out_csv, delimiter='\t')
    #
    #   with in_csv:
    #
    #       reader = csv.reader(in_csv, delimiter='\t')
    #       for row in reader:


if __name__ == '__main__':
    main()

Edit: Using Python 2.7.2

Edit: Draft code:

input_file = 'in_file.csv'
output_file = 'out_file.csv'


def main():

    try:
        with open(input_file, 'rb') as in_csv, open(output_file , 'wb') as out_csv:
            writer = csv.writer(out_csv, delimiter='\t')
            reader = csv.reader(in_csv, delimiter='\t')
            for row in reader:
                # continue processing
                # many lines of code...
    except IOError as e:
        print "Error: cannot open {0}".format(file)
        if e.errno == errno.EACCES:
            print "\tPermission denied."
            print "\tError message: {0}".format(e)
            sys.exit()
        # Not a permission error.
        print "\tDoes file exist?"
        print "\tError message: {0}".format(e)
        sys.exit()


if __name__ == '__main__':
    main()

My draft code feels a bit bloated inside the try statement (imagine 100 additional lines of code). Is there a better approach?

this should probably go on codereview.statckoverflow.com – Joran Beasley Sep 16 '13 at 15:52 — Joran Beasley, Sep 16 '13 at 15:52
@Joran Beasley: thanks, will keep this in mind from now on. – fire_water Sep 16 '13 at 17:55 — fire_water, Sep 16 '13 at 17:55

score 1 · Accepted Answer · answered Sep 16 '13 at 15:52

1

You can do it all very easily like this:

input_file = 'in_file.csv'
output_file = 'out_file.csv'

with open(input_file, 'rb') as in_csv, open(output_file , 'wb') as out_csv:
    # do your code

answered Sep 16 '13 at 15:52

Inbar Rose

41,843
24
85
131

it think this is only py2.7+ (not sure what version OP has) – Joran Beasley Sep 16 '13 at 16:14
@Inbar: to handle file access errors (i.e. a file cannot be opened), should I use a try/except ? Please see "Draft code" section of my post. – fire_water Sep 16 '13 at 18:01

score 1 · Answer 2 · answered Sep 16 '13 at 16:02

While @Inbar's answer is simple, and works well, you might want to be fancy and implement your own context manager:

import csv
input_file = 'in_file.csv'
output_file = 'out_file.csv'


class csv_io:

    def __init__(self, input_name, output_name):
        # Open files in binary read/write mode for platform independence.
        self.input = open(input_name, 'rb')
        self.output = open(output_name, 'wb')

    def __enter__(self):
        return self

    def __exit__(self, *args):
        if hasattr(self, 'input'):
            self.input.close()
        if hasattr(self, 'output'):
            self.output.close()


def main():

    with csv_io(input_file, output_file) as data:
        writer = csv.writer(data.output, delimiter='\t')
        reader = csv.reader(data.input, delimiter='\t')
        for row in reader:
            do_stuff()

    # ...and here they are closed


if __name__ == '__main__':
    main()

Much appreciated. Taught me a lot about OOP too. – fire_water Sep 16 '13 at 17:57 — fire_water, Sep 16 '13 at 17:57

score 0 · Answer 3 · answered Sep 16 '13 at 15:55

To answer your first point, you were indeed right, with open as is the way to go. The reason for this is that it makes sure that the file pointer is properly closed when exiting the with statement. In your exemple, if somewhere in your main function an unhandled Exception is raised, your script will exit without closing the file, which is bad.

For the second question, I think you should manage the Exception in the main function, in order to facilitate understanding of your code.

Open Multiple Files, Helper Function

3 Answers3