Overhead of passing around a file object instead of file name?

Question

I have a method that detects what file it should open based on input, opens the file, then returns the file object.

def find_and_open(search_term):
    # ... logic to find file
    return open(filename, 'r')

I like this way because it hides the most amount of implementation from the caller. You give it your criteria, it spits out the file object. Why bother with String paths if you're just going to open it anyway?

However, in other Python projects I tend to see such a method return a string of the filepath, instead of the fileobject itself. The file is then opened at the very last minute, read/edited, and closed.

My questions are:

from a performance standpoint, does passing around file objects carry more overhead? I suppose a reference is a reference no matter what it points to, but perhaps there's something going on in the interpreter that makes a String reference faster to pass around than a file reference?
From a purely "Pythonic" standpoint, does it make more sense to return the file object, or the String path (and then open the file as late as possible)?

Why don't you use the `timeit` or `profile` modules and find out for yourself? — martineau, Jul 15 '15 at 03:08
I prefer to return the String path, and open file using `with open(...) as ...` — LittleQ, Jul 15 '15 at 03:15

Kyler Brown · Accepted Answer · 2015-07-15T04:01:53.403

Performance is unlikely to be an issue. Reading and writing to disk are orders of magnitude slower than reading from RAM, so passing a pointer around is very unlikely to be the performance bottleneck. 1
From the python docs:

It is good practice to use the with keyword when dealing with file objects. This has the advantage that the file is properly closed after its suite finishes, even if an exception is raised on the way. It is also much shorter than writing equivalent try-finally blocks ...

Note that you can both use with to open a file and pass the file object around to other functions, either by nesting functions or using yield. Though I would consider this less pythonic than passing a file string around in most cases.

Simple is better than complex. Flat is better than nested.

You might also be interested in pathlib.

Overhead of passing around a file object instead of file name?

1 Answers1