1

I have a very simple script which just reads in a file, that somehow does now work when I run it from inside Sublime Text, but works fine when I invoke Python from the command line. I wonder why?

It seems that in open() in one case the default encoding is utf-8, but not in the other case. Why would that be? The Python executable is the same for both, and both seem to have the same core Python libraries in the path.

Script:

import sys
print(sys.executable)
print(sys.path)
open('foo.txt').read()
print('Life is wonderful.')

Output when run from command line after activating virtualenv:

/Users/bemmu/Dropbox/b2/babynames/env/bin/python3
['/Users/bemmu/Dropbox/b2/babynames', '/Users/bemmu/Dropbox/b2/babynames', '/Users/bemmu/b2/python_include_dir', '/Library/Python/2.5/site-packages/elementtree-1.2.7_20070827_preview-py2.5.egg', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/Users/bemmu/Dropbox/b2/babynames/env/lib/python3.6/site-packages', '/Users/bemmu/Dropbox/b2/babynames/env/lib/python3.6/site-packages/fasttext-0.9.2-py3.6-macosx-10.6-intel.egg']
Life is wonderful.

Output when run from Sublime Text 3:

/Users/bemmu/Dropbox/b2/babynames/env/bin/python3
['/Users/bemmu/Dropbox/b2/babynames', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/Users/bemmu/Dropbox/b2/babynames/env/lib/python3.6/site-packages', '/Users/bemmu/Dropbox/b2/babynames/env/lib/python3.6/site-packages/fasttext-0.9.2-py3.6-macosx-10.6-intel.egg']
Traceback (most recent call last):
  File "/Users/bemmu/Dropbox/b2/babynames/foo.py", line 4, in <module>
    open('foo.txt').read()
  File "/usr/local/bin/../../../Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128)
[Finished in 0.0s with exit code 1]
[cmd: ['env/bin/python3', '/Users/bemmu/Dropbox/b2/babynames/foo.py']]
[dir: /Users/bemmu/Dropbox/b2/babynames]
[path: /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/usr/local/go/bin:/opt/X11/bin:/Library/Apple/usr/bin:/Applications/Wireshark.app/Contents/MacOS:/usr/local/git/bin]

Build system:

{
    "cmd": ["env/bin/python3", "$file"],
    "selector": "source.python",
    "file_regex": "^\\s*File \"(...*?)\", line ([0-9]*)"
}
Bemmu
  • 17,849
  • 16
  • 76
  • 93
  • try opening with specified encoding `, encoding="utf-8")` – Sin Han Jinn Oct 09 '20 at 08:50
  • There is like a whole [discussion](https://stackoverflow.com/questions/10561923/unicodedecodeerror-ascii-codec-cant-decode-byte-0xef-in-position-1) here talking about this. Might explain things. – Sin Han Jinn Oct 09 '20 at 08:58
  • 1
    @HjSin Thanks. That discussion was helpful in solving the problem. I posted an answer, but if you want you can post the same answer and I'll accept it. – Bemmu Oct 09 '20 at 09:04

1 Answers1

0

You need to set the LANG environment variable in your build system.

For example:

{
    "env": {"LANG":"en_US.UTF-8"},
    "cmd": ["env/bin/python3", "$file"],
    "selector": "source.python",
    "file_regex": "^\\s*File \"(...*?)\", line ([0-9]*)"
}
Bemmu
  • 17,849
  • 16
  • 76
  • 93