Make IPython Import What I Mean

Question

I want to modify how IPython handles import errors by default. When I prototype something in the IPython shell, I usually forget to first import os, re or whatever I need. The first few statements often follow this pattern:

In [1]: os.path.exists("~/myfile.txt")
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-0ffb6014a804> in <module>()
----> 1 os.path.exists("~/myfile.txt")

NameError: name 'os' is not defined

In [2]: import os

In [3]: os.path.exists("~/myfile.txt")
Out[3]: False

Sure, that's my fault for having bad habits and, sure, in a script or real program that makes sense, but in the shell I'd rather that IPython follow the DWIM principle, by at least trying to import what I am trying to use.

In [1]: os.path.exists("~/myfile.txt")
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-0ffb6014a804> in <module>()
----> 1 os.path.exists("~/myfile.txt")

NameError: name 'os' is not defined

Catching this for you and trying to import "os" … success!
Retrying …
---------------------------------------------------------------------------
Out[1]: False

If this is not possible with a vanilla IPython, what would I have to do to make this work? Is a wrapper kernel the easiest way forward? Or should this be implemented directly in the core, with a magic command?

Note, this is different from those kind of question where someone wants to always load pre-defined modules. I don't. Cuz I don't know what I will be working on, and I don't want to load everything (nor do I want to keep the list of everything updated.

Aaron Christiansen · Accepted Answer · 2016-03-22T16:51:39.350

11

NOTE: This is now being maintained on Github. Download the latest version of the script from there!

I developed a script that binds to IPython's exception handling through set_custom_exc. If there's a NameError, it uses a regex to find what module you tried to use and then attempt to import it. It then runs the function you tried to call again. Here is the code:

import sys, IPython, colorama # <-- colorama must be "pip install"-ed

colorama.init()

def custom_exc(shell, etype, evalue, tb, tb_offset=None):
    pre = colorama.Fore.CYAN + colorama.Style.BRIGHT + "AutoImport: " + colorama.Style.NORMAL + colorama.Fore.WHITE
    if etype == NameError:
        shell.showtraceback((etype, evalue, tb), tb_offset) # Show the normal traceback
        import re
        try:
            # Get the name of the module you tried to import
            results = re.match("name '(.*)' is not defined", str(evalue))
            name = results.group(1)

            try:
                __import__(name)
            except:
                print(pre + "{} isn't a module".format(name))
                return

            # Import the module
            IPython.get_ipython().run_code("import {}".format(name))
            print(pre + "Imported referenced module \"{}\", will retry".format(name))
        except Exception as e:
            print(pre + "Attempted to import \"{}\" but an exception occured".format(name))

        try:
            # Run the failed line again
            res = IPython.get_ipython().run_cell(list(get_ipython().history_manager.get_range())[-1][-1])
        except Exception as e:
            print(pre + "Another exception occured while retrying")
            shell.showtraceback((type(e), e, None), None)
    else:
        shell.showtraceback((etype, evalue, tb), tb_offset=tb_offset)

# Bind the function we created to IPython's exception handler
IPython.get_ipython().set_custom_exc((Exception,), custom_exc)

You can make this run automatically when you start an IPython prompt by saving it somewhere and then setting an environment variable called PYTHONSTARTUP to the path to this file. You set environment variables differently depending on your OS (remember to alter the paths):

Windows: setx PYTHONSTARTUP C:\startup.py in command prompt
Mac/Linux (bash): Put export PYTHONSTARTUP=$HOME/startup.py into your ~/.bashrc

Here's a demo of the script in action:

Demo

edited Mar 22 '16 at 16:51

answered Mar 20 '16 at 16:12

Aaron Christiansen

11,584
5
52
78

_Currently, this script infinite-loops on some errors_ - if import results in NameError and clean-up routine does same import... - you already know what happens. You need to check if module you're trying to import exists. – Łukasz Rogalski Mar 20 '16 at 17:38
@Rogalski How would I do that? I know `pip` has methods to do this but somebody might be importing something that's on their local machine. – Aaron Christiansen Mar 20 '16 at 17:47
Run: `try: import whatever except ImportError: oops_failed_to_import_handle_it()`? – Łukasz Rogalski Mar 20 '16 at 18:39
@Rogalski Thanks, I'm updating it to support this, as well as have clearer output. – Aaron Christiansen Mar 20 '16 at 18:42
@Rogalski This has now been updated to look prettier and not infinite loop. – Aaron Christiansen Mar 20 '16 at 19:00
Nice, a IPython exception handler seems appropriate! Why did you just pass `NameError` as [first parameter for set_custom_exc](http://ipython.readthedocs.org/en/stable/api/generated/IPython.core.interactiveshell.html?highlight=set_custom_exc#IPython.core.interactiveshell.InteractiveShell.set_custom_exc)? Finally, would there be a way to distinguish `NameError`s that occur in the *library or program code* from the ones in the shell? If my program code has a bug, resulting in such a NameError, then trying to import the variable name will only make things worse. – mknecht Mar 22 '16 at 14:13
@mknecht Oh, I'll look into `set_custom_exc`'s parameters. I don't think there's a way of detecting where an error came from, but I'll look into it. – Aaron Christiansen Mar 22 '16 at 14:17
I think, we'd have to do some stack trace analysis or so … anyway, your answer is the way forward I think. – mknecht Mar 22 '16 at 14:20
@OrangeFlash81 would you consider putting your code snippet under a permissive license such as BSD or MIT or so? Would love to see this finished up, and made usable for more people. (pip installable or so) – mknecht Mar 22 '16 at 14:22
@mknecht Sure! Later I'll make a Github repo under the MIT license and post it here. – Aaron Christiansen Mar 22 '16 at 15:23

Make IPython Import What I Mean

1 Answers1