How to format Python when using for loop inside with block

Question

I've only recently starting using with to open files, instead of the more old-school separate open/close calls.

However, I'm finding this means all my code to iterate through files now has double-indentation:

with open('filename', 'rb') as f:
    for line in f:
        # do stuff

which is kinda ugly, compared to:

f = open('filename', 'rb')
for line in f:
    # do stuff
f.close()

I've tried:

with open('filename', 'rb') as f:
  for line in f:
    # do stuff

so that the "stuff" is only single indented, but this makes various lint checkers complain. Is there a better way of handling this?

Note: I appreciate the extra benefits of context managers, I'm just after the best way to format the code.

In my opinion, the best way is your original double indentation. — jpp, Jun 25 '18 at 09:01
Is it enough to reduce the indentation levels with more fine-grained functions? — Moberg, Jun 25 '18 at 09:01
Why is the first one "kinda ugly"? If you're using Python, you need to get used to multiple levels of indentation. — Daniel Roseman, Jun 25 '18 at 09:04
It's "kinda ugly" because I'm very unlikely to have anything else at the first indentation level, so I've "wasted" an indentation. It just makes following the flow of code harder, as it's one logical thing "go through this file one line at a time" but it's using 2 indentations. — xorsyst, Jun 25 '18 at 15:49

DeepSpace · Answer 1 · 2018-06-25T09:16:16.067

4

You don't have to put the logic inside the with block:

with open('filename', 'rb') as f:
    file_content = f.readlines()

for line in file_content:
    # do stuff

The downside with this approach is that the whole file would need to be saved into the file_content list.

You can still have the benefit of the generator if you hide the read logic in a separate function:

def read_file(file_path):
    with open(file_path, 'rb') as f:
        for line in f:
            yield line
        # or simply `yield from f` in Python 3

for line in read_file(file_path):
    # do stuff

BUT all this is probably a total overkill for something as trivial as a two level indention. You will be better off learning to live with it.

edited Jun 25 '18 at 09:16

answered Jun 25 '18 at 09:03

DeepSpace

78,697
11
109
154

1

You can remove that for-loop using yield from: `with open(file_path, 'rb') as f: yield from f` – Maxime Chéramy Jun 25 '18 at 09:11
@MaximeChéramy Indeed, I'll add it as a comment since it only works in Python 3 – DeepSpace Jun 25 '18 at 09:15

score 1 · Answer 2 · answered Jun 25 '18 at 09:28

To me, your original code is perfectly fine:

with open('filename', 'rb') as f:
    for line in f:
        # do stuff

However, if #do stuff becomes too big, I encourage you to use a function:

def do_stuff(f):
    for line in f:
        # do stuff

with open('filename', 'rb') as f:
    do_stuff(f)

When performance isn't an issue, it's very common to store the content of the file in a variable:

with open('filename', 'rb') as f:
    lines = f.readlines()

for line in lines:
    # do stuff

This is more useful when you need to manipulate several files at the same time and that you do not wish to nest several with structures.

"When performance isn't an issue, it's very common to store the content of the file in a variable" => when memory is not an issue either... — bruno desthuilliers, Jun 25 '18 at 09:32

serv-inc · Answer 3 · 2018-06-25T09:17:07.067

@Maxime's comment just beats it all:

def read_file(file_path):
    with open(file_path, 'rb') as f: 
        yield from f

for line in read_file('filename'):
    # do sth

This uses Python's yield from, explained at In practice, what are the main uses for the new "yield from" syntax in Python 3.3?.

And to validate your concerns, here is the Zen of Python (import this)

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

It really gets ugly at four (?) and more levels of indentation. In that case, you might be well-advised to clean this up, and move the whole file-handling code to a function, like so:

with open('filename', 'rb') as f:
    parse_file(f)

def parse_file(f):
    for line in f:
        # do stuff

It gets really deep when you start adding classes etc to the mix ....

Apart from that: do you have a nice editor, that eases the indentation etc? While not for everyone, emacs is actually quite good with Python

How to format Python when using for loop inside with block

3 Answers3