2

I have written a class like this:

class FooBar(object):
    # some methods
    # ...
    def __enter__(self, param1, param2):
        # do something here ...
        pass

I try to use my class like this (imported from module mymod):

with (mymod.FooBar("hello", 123)) as x:
    # do something here with instance of mymod.FooBar called x ...
    pass

when the block above gets executed, I get the error:

__enter__() takes exactly 3 arguments (1 given)

What am I doing wrong?

Homunculus Reticulli
  • 65,167
  • 81
  • 216
  • 341

3 Answers3

12

The __enter__ method is never given any arguments, so beyond self your signature should not have any other.

You should move those arguments to the __init__ method instead:

class FooBar(object):
    def __init__(self, param1, param2):
        # do something here ...

    def __enter__(self):
        # something else, perhaps return self

Creating an instance of FooBar() is a separate step. with calls __enter__ on the result of your mymod.FooBar("hello", 123) expression, the expression itself is not translated to an __enter__ call.

If it was, you couldn't also use it like this, but you can:

cm = mymod.FooBar("hello", 123)
with cm as x:
    # do something here with x, which is the return value of cm.__enter__()

Note that x is assigned whatever cm.__enter__() returned; you can return self from __enter__ or you can return something entirely different.

The expected methods __enter__ and __exit__ are documented in the With Statement Context Managers section of the Python Data model documentation:

object.__enter__(self)

Enter the runtime context related to this object. The with statement will bind this method’s return value to the target(s) specified in the as clause of the statement, if any.

as well as in the Content Manager Types section of the Built-In Types documentation:

contextmanager.__enter__()

Enter the runtime context and return either this object or another object related to the runtime context. The value returned by this method is bound to the identifier in the as clause of with statements using this context manager.

An example of a context manager that returns itself is a file object. File objects return themselves from __enter__() to allow open() to be used as the context expression in a with statement.

If you are interested in the exact interactions, see the original proposal: PEP 343 -- The "with" Statement; from the specification section you can see what with EXPR as VAR: BLOCK statement does under the hood:

mgr = (EXPR)
exit = type(mgr).__exit__  # Not calling it yet
value = type(mgr).__enter__(mgr)
exc = True
try:
    try:
        VAR = value  # Only if "as VAR" is present
        BLOCK
    except:
        # The exceptional case is handled here
        exc = False
        if not exit(mgr, *sys.exc_info()):
            raise
        # The exception is swallowed if exit() returns true
finally:
    # The normal and non-local-goto cases are handled here
    if exc:
        exit(mgr, None, None, None)

Note the mgr = (EXPR) part; in your case, mymod.FooBar("hello", 123) is that part. Also note that (EXPR), __enter__ and __exit__ are not 'protected' by the try..except here, exceptions raised in the expression or when entering or exiting are not handled by the context manager!

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • But I need to pass params to the ctor (constructor), in order to create a valid object – Homunculus Reticulli Apr 24 '15 at 13:23
  • 4
    @HomunculusReticulli: yes, but `__enter__` is *not the constructor*. – Martijn Pieters Apr 24 '15 at 13:24
  • Given, your very high score, may I be so bold to ask you if you're sure that this is the case. I have always thought (read somewhere?) that the with statement **instantiates ** the object (in otherwords it was just syntactic sugar for the ctor), likewise the __exit__ was analogous to the dtor. It would seem that large tracts of my code are wrong then, if this is the case. – Homunculus Reticulli Apr 24 '15 at 13:27
  • You can still write `with mymod.FooBar("hello", 123)` but you must use the params in the `__init__` method, not in the `__enter__` method. – Jérôme Apr 24 '15 at 13:27
  • 3
    @HomunculusReticulli: I am absolutely, positively, 100% sure, yes. – Martijn Pieters Apr 24 '15 at 13:28
  • @HomunculusReticulli: No. The `with` statement is not a constructor. Its purpose is to ensure certain code runs when you enter or leave a given scope by any means. – Kevin Apr 24 '15 at 13:28
  • @MartijnPieters: ok. In my ctor, I'm doing a LOT of stuff, checking for existence of a database file if not exist, verifying schema etc. If something goes wrong during the instantiation, I don't want to end up with lots of corrupt database files. Thats why I have always used with and exit. The code you show will work, but if an exception is thrown in the construction, my exit won't be called, so no cleaning up will be done. I'm VERY confused now. – Homunculus Reticulli Apr 24 '15 at 13:36
  • @HomunculusReticulli: the context manager is meant to handle exceptions *in the `with` block*, not in the constructor, or the `__enter__` methods. The goal here is to clean up the context if and when the block exits, not during creation of the manager or when setting up the manager for a given block. – Martijn Pieters Apr 24 '15 at 13:40
  • @HomunculusReticulli: compare this to opening a file. The file object context manager can only be used once you have the open file, problems with opening the file are outside the scope there. – Martijn Pieters Apr 24 '15 at 13:41
  • What does the `__enter__()` method of a file do ? Just return the file instance ? – Jérôme Apr 24 '15 at 13:48
  • @Jérôme: yes, all it does is `return self`; the [`__enter__` method slot](https://hg.python.org/cpython/file/0ac30526c208/Objects/fileobject.c#l2135) links to the [`file_self` function](https://hg.python.org/cpython/file/0ac30526c208/Objects/fileobject.c#l1996); if the file is closed an exception is raised instead. – Martijn Pieters Apr 24 '15 at 13:49
  • So `file` is a trivial case of a more complex mechanism where `__enter__()` actually does stuff? – Jérôme Apr 24 '15 at 13:50
  • +1 for the very detailed explanation. I will accept your answer, fix my code as recommended and come back to digest all this later. – Homunculus Reticulli Apr 24 '15 at 13:51
  • @Jérôme: exactly. `sqlite3` connection objects start a transaction, `MySQLdb` connection objects start a transaction and return a new cursor object, for example. – Martijn Pieters Apr 24 '15 at 13:51
  • Likewise, could there be such thing as a file connection object that would open the file in `__enter__`? – Jérôme Apr 24 '15 at 13:56
  • @Jérôme: sure; just create an object that stores the arguments for `open()` for later use, and call `open()` in `__enter__`; remember to store the result as well, pass on the call to `__enter__`, and call `__exit__` when exiting again. Note that any exceptions are still not going to be handled as part of the `with` context. – Martijn Pieters Apr 24 '15 at 13:58
  • So where would you catch DB opening exceptions, for instance, if they occur in `__enter__` in the with statement? (I hope asking things that far from OP's question in comments is not inappropriate. I'm trying to clarify this example as it illustrates the use of `with`.) – Jérôme Apr 24 '15 at 14:48
  • @Jérôme: you'd use `try...except`, or delegate that to another context manager. – Martijn Pieters Apr 24 '15 at 14:49
  • @Jérôme: in short, a context manager manages a block of code; the manager is not itself part of that block. Creating the manager cannot be part of the block either. – Martijn Pieters Apr 24 '15 at 14:50
  • I think my question was the same as [this one](http://stackoverflow.com/questions/3642080/using-python-with-statement-with-try-except-block) and the answer would be to catch IOError outside of the with statement (I always thought it would be better to catch exception as close as possible to when they happen. For instance here, if an IOError occurs somewhere in the block, it'll be caught as well.) – Jérôme Apr 24 '15 at 14:58
  • @Jérôme you can always open the file separately then use it as a CM further on. – Martijn Pieters Apr 24 '15 at 15:01
  • Isn't this exactly what `with open("myfile.txt") as f:` does? – Jérôme Apr 24 '15 at 15:03
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/76187/discussion-between-martijn-pieters-and-jerome). – Martijn Pieters Apr 24 '15 at 15:08
2

Consider this code:

f = open("myfile.txt")
with f:
    x = f.read()

This is basically the same as

with open("myfile.txt") as f:
    x = f.read()

Notice how the object initialization and with context are separate things. The with context is only responsible for calling the object's enter() and exit() methods at the appropriate times.

Joel Cornett
  • 24,192
  • 9
  • 66
  • 88
  • +1 Ah, somehow, I seem to understand it, now that you show how the two blocks are semantically equivalent... My confidence however is badly shaken :( It seems I didn't know as much Python as I thought I did ... – Homunculus Reticulli Apr 24 '15 at 13:49
  • 1
    @HomunculusReticulli: They are not *quite* equivalent, but for file objects it doesn't really matter. In the first example, you can do `f.close()` in between and then you'd get an exception when using `with f:`, for example. But `f.__enter__()` returns `f` so apart from that the two snippets are basically equivalent. – Martijn Pieters Apr 24 '15 at 14:00
0

mymod.FooBar() is calling the __init__ method of the FooBar class and returning an instance of the class as an object.

Your __enter__ method should only accept self

enigma
  • 3,476
  • 2
  • 17
  • 30