How future imports work under the hood

Question

I've been fascinated by the __future__ module - in particular, its ability to change the way statements are parsed in python.

What's most interesting is how doing something like

from __future__ import print_function

Enables you to use print (and not print_function, like you would expect any other normal import to do).

I have read What is __future__ in Python used for and how/when to use it, and how it works thoroughly and in particular came across a particular line:

A future statement is a directive to the compiler that a particular module should be compiled using syntax or semantics that will be available in a specified future release of Python.

I would love to know the intricacies of what exactly makes this possible. In particular, how something like

from __future__ import division

Can enable true division on python2, while

from __future__ import barry_as_FLUFL

Can enable the <> syntax on python3 (what I find most funny is that you have to import a feature from "__future__" for backward compatibility).

Anyway, to summarise, I would like to know how the directive is understood and executed by the compiler when __future__ or its artefacts are imported.

Because it's *not* a normal import. See https://docs.python.org/2/reference/simple_stmts.html#future — jonrsharpe, Aug 21 '17 at 16:08
@jonrsharpe "_Anyway, to summarise, I would like to know how the directive is understood and executed by the compiler when __future__ or its artefacts are imported._" I have done my research but thanks for the downvote anyway. — cs95, Aug 21 '17 at 16:10
I should've probably changed the title to discourage people with short attention spans from down voting. — cs95, Aug 21 '17 at 16:12
I feel like there is definitely some _meat_ to this question for whatever my opinion is worth. — Ma0, Aug 21 '17 at 16:15
@Ev.Kounis Those 3 downvoters and 1 close voter would disagree, but whatever, like I said... probably my fault for the initial bad title. — cs95, Aug 21 '17 at 16:17
The `__future__` module has nothing to do with this behavior. — user2357112, Aug 21 '17 at 16:19
@user2357112 Can you elaborate.. (in an answer maybe)? The `__future__` import would trigger those various flags, at least, that's what I've understood. — cs95, Aug 21 '17 at 16:21
to me it's like Perl's `use strict` or `use warnings` which change some internal flags which in turn would change the language's behavior. — pynexj, Aug 21 '17 at 16:26

Martijn Pieters · Accepted Answer · 2017-08-21T17:07:49.290

from __future__ import print_function tells the parser to not treat print as a keyword (leaving it as a name instead). That way the compiler treats it as the function and not a statement.

To track this, the compiler struct has a c_future field that holds a PyFutureFeatures object that tracks which future directives have been enabled. Various parts of the parser and compiler check the flags and alter behaviour.

This is mostly handled in the future.c source file, which has a future_parse() function that checks for import from AST objects with the module parameter set to __future__, and sets flags based on what is found.

For example, for the barry_as_FLUFL 'feature', the parser refuses != as syntax but accepts <> instead:

if (type == NOTEQUAL) {
    if (!(ps->p_flags & CO_FUTURE_BARRY_AS_BDFL) &&
                    strcmp(str, "!=")) {
        PyObject_FREE(str);
        err_ret->error = E_SYNTAX;
        break;
    }
    else if ((ps->p_flags & CO_FUTURE_BARRY_AS_BDFL) &&
                    strcmp(str, "<>")) {
        PyObject_FREE(str);
        err_ret->text = "with Barry as BDFL, use '<>' "
                        "instead of '!='";
        err_ret->error = E_SYNTAX;
        break;
    }
}

You can find the other examples by grepping for the FUTURE_* flags listed in compile.h.

Note that there is a __future__ Python module, but it is not directly involved in the parsing and compilation of code; it is merely there to give Python code easy access to metadata about directives (including bitfield values to pass to the flags argument of the compile() function), nothing more.

I know this already. What I wanted to know how it works under the hood... that was the purpose of this question. — cs95, Aug 21 '17 at 16:10
@cᴏʟᴅsᴘᴇᴇᴅ: that's.. rather broad, but I'll see what I can fill in. — Martijn Pieters, Aug 21 '17 at 16:12
What I find most funny about barry is that he has to come from `__future__` instead of something more sensible like `__past__`. Thanks for the great answer as always. — cs95, Aug 21 '17 at 16:30

How future imports work under the hood

1 Answers1

Linked

Related

How __future__ imports work under the hood

1 Answers1

Linked

Related

How future imports work under the hood