2

Despite all that has been said and written on Python2 vs Python3, I have been unable to identify why the developers made it impossible to mix Python2 and Python3 code. Surely there must be a reason for this?

In Fortran, for instance, the many versions are incompatible with each other, but they can still happily co-exist within the same project. The same applies to C and C++: some C code is not compatible with C++, but the compiler is able to recognize the correct language using the file extension. Is there a specific reason for why this approach was not chosen for Python3? That is, let Python3 modules be identified by a .py3 extension (or a shebang comment), and use one single interpreter for both .py and .py3 code?

EDIT:

There is already a question named Why is Python 3 not backwards compatible? , but this question is different. I know that Python 3 introduces new features and breaks backwards compatibility because of this. It still does not mean that Python 2 and 3 cannot coexist the same way C and C++ can.

akvilas
  • 586
  • 2
  • 5
  • 15
  • 1
    Possible duplicate of [Why is Python 3 not backwards compatible?](https://stackoverflow.com/questions/9066956/why-is-python-3-not-backwards-compatible) – Thomas Jun 07 '17 at 07:37
  • 2
    @Thomas Nope, not what he's asking. Backwards compatibility =/= coexistence of two different versions – Markus Meskanen Jun 07 '17 at 07:40
  • True, it's subtly different, so maybe not appropriate for a close vote. But backwards compatibility is a prerequisite for coexistence, so it does answer the question. – Thomas Jun 07 '17 at 07:42
  • *"backwards compatibility is a prerequisite for coexistence"* that sounds more like the answer for @akvilas's question than the "duplicate" you've linked. Also you might need some source or explanation on that one, since OP explicitly said: *"In Fortran, for instance, the many versions are incompatible with each other, but they can still happily co-exist within the same project."* – Markus Meskanen Jun 07 '17 at 07:44
  • From `Python2` to `Python3`, with more lines of code, with more imports and maybe with more `try.. except` or `if ... else` blocks you can create a script that can run well in both `Python2 and Python3`. However, i cannot see any reason to import a code from `Python3` to `Python2` ? – Chiheb Nexus Jun 07 '17 at 07:45
  • @ChihebNexus You can absolutely write code that is compatible with both versions. But why does there not exist an interpreter that can understand both versions? Why can't I use my old Python 2 library with the Python 3 interpreter? – akvilas Jun 07 '17 at 07:52
  • To my understanding, I'm not sure it's worth posting an answer, the big difference comes from the fact that python doesn't generate machine code like what you get when compiling any C or C++ source. This machine code goes straight to the processor which doesn't care from which version of which language it was created. On the other hand python compiles to a byte code, which is then interpreted and executed by a virtual machine (written in C). This bytecode is absolutely not machine code. Therefore the way the vm interprets it is highly dependent on the version associated with the bytecode. – Anis Jun 07 '17 at 07:52
  • And another proof of that is that the bytecode generated by python's compiler, contains a "magic number" which indicated the version of the bytecode. Therefore if you compile a python2 code and try to use the compiled file in an import in a script executed with python3 you'll get the sweet ImportError: bad magic number – Anis Jun 07 '17 at 07:54
  • @akvilas, not in most cases the code is compatible between python3 and python2. But, it's true you can reach this compatiblity in somehow. But, be sure of one thing. Python2 and 3's interpretors are different, even more their parsers. And this will break their compatibility. – Chiheb Nexus Jun 07 '17 at 07:56
  • Yes, the interpreters are different and incompatible. But why? If Intel can write Fortran compilers that understand all versions back to 1950, then surely Guido & co can write an interpreter that understands both python 2 and 3? – akvilas Jun 07 '17 at 07:58
  • 3
    Python 2 and 3 don’t have compatible standard libraries, so you can’t pass objects back and forth between them trivially. You could make an interpreter that ran files as one or the other, but you can already do that by installing both interpreters and writing a small shell script that picks one based on the extension. No point in making the entirety of both interpreters mandatory for people who only need to run one. – Ry- Jun 07 '17 at 08:04
  • Incompatible standard libraries: That's the best argument I've heard so far. I can easily imagine that maintaining a distinction between, say, Python2 and Python3 strings would be a hazzle. Still, converting all existing libraries to Python3 is also a hazzle.... And C/C++ have incompatible standard libraries as well. – akvilas Jun 07 '17 at 08:15
  • @akvilas: C and C++ don't have incompatible standard libraries. You can use the C standard libraries from C++ just fine. The reverse isn't true. However, C++ does not have a standard ABI, which complicates things, with C++ you can't reliably mix different compilers for a project. – Arafangion Jun 08 '17 at 02:16

2 Answers2

2

You can't mix python2 and python3 in the same project, because:

  1. The interpreters are different. You're either running an interpreter for python2, or for python3. I don't know of any current interpreter that dynamically chooses the python2 or python3 runtime.
  2. The syntax is (slightly) different.
  3. The types are different.

However, you could certainly run both the python2 and python3 runtimes and use some sort of (IPC) message passing between them.

With the case of C, and C++, you can run them in the same process, so that's fine. Incidentially, you can run python and C (or C++) in the same process, too.

The only way I can think of that would allow you to run python2 and python3 in the same process would be to embed both runtimes in the same process, however, they will very likely clobber each other's globals and get confused.

Arafangion
  • 11,517
  • 1
  • 40
  • 72
  • You are describing the current state of affairs. I'm interested in your reason (1), and ask _why_ this is the case. Is it technically difficult (for the developers of python) to create an interpreter that chooses between python 2 and 3 dynamically? – akvilas Jun 07 '17 at 07:48
  • Well, it requires work to do so, for no benefit. The whole point of doing a python3, was to have the freedom to make changes without preserving backwards compatibility. – Arafangion Jun 07 '17 at 07:57
  • Yes, it would result in a slower, much more complicated interpreter. The whole reason to move to Python3 was to free it from backwards compatibility concerns. – OldGeeksGuide Jun 07 '17 at 07:58
  • Slower and more complicated interpreter: Point taken, if it is true. But how can it be slower? You check the file name, then choose the interpreter. Easy..? – akvilas Jun 07 '17 at 08:01
  • Yes, that's already the case on all systems except windows. (The shebang line chooses the interpreter). On windows, some people are using a python "chooser" application that emulates this. – Arafangion Jun 07 '17 at 08:03
  • @Arafangion I can do this if my code consists of isolated scripts, but not if i need Python3 code to call a Python2 library. – akvilas Jun 07 '17 at 08:16
  • @akvilas: You can either refactor that code to run on either interpreter (using six, future, whatever), or refactor the code to run on the newer interpreter and step away from python2. – Arafangion Jun 07 '17 at 08:19
  • Yeah, that is the right approach given the current situation. But I'm asking why we are in this situation in the first place. – akvilas Jun 07 '17 at 08:21
  • Because python2 has a lot of warts and gotcha's, and it was collectively decided that a clean break from the past was a Good Thing. Today, that is much more controversial, but as they say, hindsight is 20/20. – Arafangion Jun 08 '17 at 00:29
1

Is there a specific reason for why this approach was not chosen for Python3? That is, let Python3 modules be identified by a .py3 extension (or a shebang comment), and use one single interpreter for both .py and .py3 code?

Because it's completely unnecessary! With the help of modules like six, it's quite easy to write code which is compatible with both Python2 and Python3 with no source-code changes. This isn't just a parlour trick; major projects like Django have been written this way.

  • I'm not talking about how to write new code. Surely it is possible to write code that both python2 and python3 interpreters can understand. – akvilas Jun 07 '17 at 07:49
  • 2
    Then you would end up with a more complicated python3 interpreter. Part of the reason Python3 was created was to avoid 'mistakes' from python2. Carrying it all forward would result in a slower bulkier interpreter. In theory, yes you could do that, but it's not desirable or necessary. – OldGeeksGuide Jun 07 '17 at 07:56