313

Suppose you have three objects you acquire via context manager, for instance A lock, a db connection and an ip socket. You can acquire them by:

with lock:
   with db_con:
       with socket:
            #do stuff

But is there a way to do it in one block? something like

with lock,db_con,socket:
   #do stuff

Furthermore, is it possible, given an array of unknown length of objects that have context managers, is it possible to somehow do:

a=[lock1, lock2, lock3, db_con1, socket, db_con2]
with a as res:
    #now all objects in array are acquired

If the answer is "no", is it because the need for such a feature implies bad design, or maybe I should suggest it in a pep? :-P

martineau
  • 119,623
  • 25
  • 170
  • 301
olamundo
  • 23,991
  • 34
  • 108
  • 149
  • I've started [a meta discussion about reversing the duplicate target for this question](https://meta.stackoverflow.com/q/378616/8117067). – Graham Jan 06 '19 at 06:04
  • @timgeb I've [started a discussion about this duplicate closure](https://meta.stackoverflow.com/q/378616/8117067). My apologies for not @-notifying you sooner; it slipped my mind. – Graham Jan 06 '19 at 13:55

4 Answers4

524

In Python 2.7 and 3.1 and above, you can write:

with A() as X, B() as Y, C() as Z:
    do_something()

This is normally the best method to use, but if you have an unknown-length list of context managers you'll need one of the below methods.

In Python 3.3, you can enter an unknown-length list of context managers by using contextlib.ExitStack:

with ExitStack() as stack:
    for mgr in ctx_managers:
        stack.enter_context(mgr)
    # ...

This allows you to create the context managers as you are adding them to the ExitStack, which prevents the possible problem with contextlib.nested (mentioned below).

contextlib2 provides a backport of ExitStack for Python 2.6 and 2.7.

In Python 2.6 and below, you can use contextlib.nested:

from contextlib import nested

with nested(A(), B(), C()) as (X, Y, Z):
    do_something()

is equivalent to:

m1, m2, m3 = A(), B(), C()
with m1 as X:
    with m2 as Y:
        with m3 as Z:
            do_something()

Note that this isn't exactly the same as normally using nested with, because A(), B(), and C() will all be called initially, before entering the context managers. This will not work correctly if one of these functions raises an exception.

contextlib.nested is deprecated in newer Python versions in favor of the above methods.

Neuron
  • 5,141
  • 5
  • 38
  • 59
interjay
  • 107,303
  • 21
  • 270
  • 254
  • thanks! So I can use this for an array of context managers with `contextlib.nested(*arr)`.
    Is this possible somehow in python 2.7 and 3.1's new syntax?     – olamundo Jun 12 '10 at 01:06
  • 4
    @noam: No, in fact the docstring for `nested` in 3.1 says: "The one advantage of this function over the multiple manager form of the with statement is that argument unpacking allows it to be used with a variable number of context managers as follows: `with nested(*managers): do_something()`" – interjay Jun 12 '10 at 09:57
  • 12
    Odd, on one hand it's deprecated, but on the other they acknowledge an advantage of the deprecated module over the replacement? – olamundo Jun 12 '10 at 19:01
  • @noam: no, [`contextlib.ExitStack`](http://docs.python.org/3/library/contextlib.html#contextlib.ExitStack) is the replacement for `contextlib.nested()`. – Martijn Pieters May 27 '13 at 14:00
  • 9
    *But usually this makes no difference*. It makes a **huge** difference if `B()` or `C()` raises an exception. The nested `with` statement will call `A().__exit__()`, but the `contextlib.nested()` setup will not! – Martijn Pieters May 27 '13 at 14:02
  • @MartijnPieters: Yes, `"usually"` was definitely too strong a word. I've clarified it. And thanks for the reference to the new `ExitStack`, I added an example. – interjay May 27 '13 at 14:32
  • +1: for adding ExitStack and completing your answer – Neil G Jun 19 '13 at 04:11
  • 11
    One issue: Using the simple "with open(A) as a, open(B) as b:" style syntax, line breaks make pep8 compliance as reported by standard tools seemingly impossible. I use a backslash to signify the line break, because surrounding the comma-separated expressions with parentheses results in the report of a syntax error. With the backslash, I get an "E127 continuation line over-indented" warning. I have yet to find a way to use this syntax while suppressing all warnings. – Darren Ringer Mar 09 '17 at 16:56
  • And any ideas on having a full replacement of contextlib.nested in Python 2.7 itself? – igordc May 29 '17 at 12:37
  • 1
    @igordcard `contextlib.nested` is still in Python 2.7, just marked as deprecated. The docs say "Developers that need to support nesting of a variable number of context managers can either use the warnings module to suppress the DeprecationWarning raised by this function or else use this function as a model for an application specific implementation.". – interjay May 29 '17 at 14:09
  • 7
    @DarrenRinger I had this same problem. I, too, used backslashes, which I abhor, to accomplish this. Double indent all the context managers after the initial with line. Only single indent the content being wrapped. It passes flake8 for me. – sage88 Sep 29 '17 at 04:40
  • 1
    @MartijnPieters Importantly, though, the `nested` call won't call the `__enter__` methods of any of the context managers if `B()` or `C()` raise an exception. That could be a *desirable* difference. This is a good reason to keep any side effects out of the initializers and put it in `__enter__`, which is what you should be doing anyway. I'd argue breaking that convention is un-Pythonic. As long as the context managers follow this convention, there's no risk involved. (interjay: I'd recommend incorporating that detail into the answer, or I could do so if you don't mind.) – jpmc26 Oct 03 '17 at 18:43
  • @jpmc26 Unfortunately, this won't help with file objects, which are probably the most common use for the `with` statement. And the same will be true for other similar objects for which using the `with` statement is optional. Also, there is another problem with `nested` mentioned in the documentation: "if the __enter__() method of one of the inner context managers raises an exception that is caught and suppressed by the __exit__() method of one of the outer context managers, this construct will raise RuntimeError rather than skipping the body of the with statement." – interjay Oct 03 '17 at 22:56
  • This answer is perfect. ExitStack is exactly what I was looking for. Thank you very much for keeping this updated. – Boris Verkhovskiy Apr 06 '20 at 22:53
83

Starting in python 3.10, you'll be able to use parenthesized context managers! Thanks @iforapsy!

with (
    mock.patch('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa') as a,
    mock.patch('bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb') as b,
    mock.patch('cccccccccccccccccccccccccccccccccccccccccc') as c,
):
    do_something()

For python versions < 3.10

@interjay's Answer is correct. However, if you need to do this for long context managers, for example mock.patch context managers, then you quickly realize you want to break this across lines. Turns out you can't wrap them in parens, so you have to use backslashes. Here's what that looks like:

with mock.patch('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa') as a, \
        mock.patch('bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb') as b, \
        mock.patch('cccccccccccccccccccccccccccccccccccccccccc') as c:
    do_something()
Neuron
  • 5,141
  • 5
  • 38
  • 59
sage88
  • 4,104
  • 4
  • 31
  • 41
  • 10
    ugly. Why not allow wrapping in parens (I got bit by this) – John Deighan Oct 23 '19 at 09:11
  • @JohnDeighan I agree, it is ugly. I also got bit by it, which is why I posted this. I hope this was helpful, though I agree, I wish it supported paren wrapping. – sage88 Oct 29 '19 at 22:41
  • @JohnDeighan Why are line breaks with \ uglier than using parens? – isarandi Nov 11 '19 at 14:45
  • 3
    To avoid the backslashes you can format it [like this](https://i.stack.imgur.com/KzuwQ.png) (had to link it as an image, as comments don't allow line breaks and this question is closed for additional answers). – Stef Mar 30 '20 at 13:36
  • 2
    mock.patch should evaluate to an object. can we also write `a_ctx = mock.patch('a'); ...; with a_ctx as a, ...:` – ThorSummoner Dec 22 '20 at 22:40
  • 3
    Good news, folks. Parenthesized context managers will be valid syntax [starting in Python 3.10, thanks to a new parser](https://docs.python.org/3.10/whatsnew/3.10.html#parenthesized-context-managers). – iforapsy Aug 05 '21 at 18:54
  • 1
    @iforapsy fantastic, I'll look into it and update my answer accordingly. – sage88 Aug 10 '21 at 21:17
  • I wish it would have separated the methods and target values - you could then assign all context managers to a single tuple. something like (TL;DR: this won't work): `with (func(), func(), func()) as contexts:` – Thomas Guyot-Sionnest Mar 25 '23 at 17:48
  • `with (*a)` still doesn't work. – HappyFace May 17 '23 at 22:04
33

The first part of your question is possible in Python 3.1.

With more than one item, the context managers are processed as if multiple with statements were nested:

with A() as a, B() as b:
    suite

is equivalent to

with A() as a:
    with B() as b:
        suite

Changed in version 3.1: Support for multiple context expressions

Mark Byers
  • 811,555
  • 193
  • 1,581
  • 1,452
  • thanks! but that still didn't answer my whole question: what about the 2nd case I mentioned, where the context managers are given in an array, without knowing how many mangers are there in the array. will it be possible in some python3.X to do `with [cm1,cm2,cm3,cm4,cm5] as result: ....` – olamundo Jun 11 '10 at 17:50
  • 2
    @noam: To solve the second part of your question you could write a class to wrap a number of resources and implement `__enter__` and `__exit__` for that class. I'm not sure if there's a standard library class that does this already. – Mark Byers Jun 11 '10 at 17:55
  • @Mark I don't think it is that easy - that's why `contextlib.nested()` is deprecated. If something happens between the generation of the other things and the activation of the context manager, it might happen that the cleanup doesn't happen as wanted. – glglgl May 27 '13 at 12:07
11

The second part of your question is solved with contextlib.ExitStack in Python 3.3.

Neil G
  • 32,138
  • 39
  • 156
  • 257