3

I have a Python PEX built using Pants 1.0.1. This PEX is a working python application, up until ubuntu 16.04 and fedora 24 released using python 2.7.12. Now I get an error indicating that the encoding of the PEX file is bad:

$ ./ainfo_py.pex 
File "./ainfo_py.pex", line 2
SyntaxError: Non-ASCII character '\xe4' in file ./ainfo_py.pex on line 3, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Duh. The zip file is embedded within the .pex file, so naturally there will be non-ascii characters in the file.

Reading the link in the error message, it says I can put an encoding at the top of the file, however, I don't think there is an encoding that would fix this, is there?

I'm not sure what to do at this point. It seems to me that python 2.7.12 broke PEX. Is there an update to pants that fixes this? Is there something I can add to the pex file to fixes this?

Edit

A little more information. I'm not sure why this ends up being important, but it seems that it does. I modify the PEX file slightly after it is built. I change the shebang at the top of the file from python2.7 to python. I want to be able to run the same PEX on both python 2.6 and 2.7 versions, based upon whatever version is available on the variety of distributions this needs to run on. It appears that if i leave it alone, as python2.7 it works fine on python 2.7.12. If i change it, it no longer works. The default python on Ubuntu 16.04 is still python 2.7, so why would changing python2.7 to python make a difference?

More Information

If I unzip the PEX, I can execute the directory and I would expect to. So I remove the first line of the PEX file, which has the #! in it, the rest is a zip. I uncompress the zip into a directory called unzipped. I can then execute python unzipped without issue. So the encoding of the source files themselves doesn't seem to be an issue. I cannot execute the zip directly. I get the same error when running python unzipped.pyz.

My Solution

So there is no really good answer to my problem here. Since I modify the resulting PEX to remove the python2.7 requirement and change it to python, to allow it to run on python 2.6 or 2.7 depending on the distribution, I'm finding that its the modification of the PEX that is causing the problem. So I've resorted to making 2 PEX files, one for python 2.7 and another for python 2.6.

In the end, I do not understand why simply touching the #! at the top of the PEX causes python to no longer understand what that file is. Maybe someone could shed some light on why that is.

KFro
  • 764
  • 3
  • 8
  • FWIW, I just created a docker image with ubuntu 16.04 and downloaded the latest pants sources and made a pex out ``` – ericzundel Jul 23 '16 at 11:03
  • I tried to reproduce your problem. I just created a docker image with ubuntu 16.04 cat /etc/issue Ubuntu 16.04 LTS \n \l python -v Python 2.7.12 (default, Jul 1 2016, 15:12:24) I downloaded the latest pants sources (1.2.0-dev0) and made a pex out of the example code ``` ./pants binary examples/src/python/example/hello/main:: ... dist/main.pex Hello, world! ``` Then, I copied the examples to another directory and installed pants 1.0.1 and was still able to get a working pex. I can't see your source, but maybe your python sources need: ``` # coding=utf-8 ``` – ericzundel Jul 23 '16 at 11:10
  • Well, its looking like something to do with the build environment I use. I have a build job on an Ubuntu 12.04 machine that creates these PEX files. I had been able to transfer these PEX's to any other python 2.6/2.7 machine and get it to work, no problem. Until Ubuntu 16.04 and Fedora 24. If I build the PEX on a 16.04 machine it works on 16.04. I have to see if building on the new 16.04 machine makes it backward compatible with older distributions. – KFro Jul 25 '16 at 17:18
  • I took the pex I built above and ran it on a different machine: CentOS release 6.5 (Final) and ran it using python 2.7.8. That same pex also worked for me on a mac (python 2.7.10). Its very basic, just the python hello `main.pex` example out of the pants OSS repo. (Oh, I just saw you added more information above.... I didn't see that before posting this) – ericzundel Jul 27 '16 at 13:17

0 Answers0