0

I am trying to use Cython to create .so binary files from our .py files and shared it with our team. However even if we all use Python3, most of times it should be exactly similar revision (let us say 3.7.8), otherwise we get an error to import them. Is this behavior expected?

Some of revisions are compatible. For example if we make .so with python 3.5.2 and import in 3.6.8 it works but it does not work in 3.7.8

Where does this mess comes from and what is the safest way to do this?

Sam
  • 1
  • 2
  • add it to requirements.txt or poetry.toml, so that it is rebuilt on install – Joran Beasley Jul 29 '20 at 22:40
  • The last digit shouldn't matter for compatibility but the others do. https://www.python.org/dev/peps/pep-3149/ should be saving you from this though - hopefully your aren't renaming your .so files to avoid it – DavidW Jul 30 '20 at 05:27
  • 1
    Some relevant questions https://stackoverflow.com/a/44324092/4657412, https://stackoverflow.com/a/54743870/4657412 – DavidW Jul 30 '20 at 05:33

1 Answers1

0

To follow up on my comments:

  • On the same platform, extension modules should work within a "minor version" (i.e. modules built with 3.7.2 and 3.7.3 should be compatible). I'm struggling to find a source for this though. Beyond that some effort has made in the past to ensure compatibility between releases, but not so much any more so it's possible you may be lucky and things work.

  • distutils/setuptools and other similar build mechanisms tag extension modules with a suffix indicating the version and some other details. For example, and extension would be called foo.cpython-37m.so instead of just foo.so. These tags prevent the module from being used with other Python versions and are a good thing. If you are removing these tags then this mess is entirely on you.

  • Python now defines a more limited stable ABI that should be compatible across Python versions. Cython is working towards supporting that but at the moment it is not in a usable state. In a year or so it should be a good solution.

In summary, .so files are not expected to be portable between different Python versions. You should either standardise on a Python version or build the .so files locally.

DavidW
  • 29,336
  • 6
  • 55
  • 86
  • Thank you guys! that was what I needed to know! I was in the impression that this .so files are compatible completely! But now I can plan better and try to not mess up things! – Sam Aug 02 '20 at 21:55