sublime and bs4 error (windows 10)

Question

I have created a virtualenv (i named it scrape), i activated it and i have installed packages and beautifulsoup4. I am trying to make sublime text 3 to work (ie show build results) within a virtualenv. I created a new build system like this:

{
    "cmd": ["C:/Users//User1/Desktop/scrape/Scripts/python.exe", "-u","$file"],
    "file_regex": "^[ ]*File \"(...*?)\", line ([0-9]*)",
    "selector": "source.python"
}

Then i run this code (i know that Betfair has an API,but this is not the point here):

import requests
from bs4 import BeautifulSoup

url = requests.get("https://www.betfair.com/")

print(url.status_code)

bs_4 = BeautifulSoup(url.text, "html.parser")

print(bs_4.prettify())

When i run the above with Powershell and virualenv activated, i am getting "200" and then all the html data (ie as ti is supposed to be). When i run inside sublime text, i am only getting "200" inside the build results panel, with no any html data. The fact that i am getting "200" means that my build system works ok, but why aren't the html results shown? Note that i am not getting any error.

Now, if i change the url from https://betfair.com to https://ubet.com.cy/sports (that contains some Greek characters), everything works ok if i do it in Powershell (ie i am getting "200" plus the html data.

import requests
from bs4 import BeautifulSoup

url = requests.get("https://ubet.com.cy/sports")

print(url.status_code)

bs_4 = BeautifulSoup(url.text, "html.parser")

print(bs_4.prettify())

If however i do it in sublime text, i am getting the "200" plus the following error:

File "C:\Users\User1\Desktop\scrape\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\x93' in position 6074: character maps to <undefined>

This is line 19:

17class IncrementalEncoder(codecs.IncrementalEncoder):
18    def encode(self, input, final=False):
19        return codecs.charmap_encode(input,self.errors,encoding_table)[0]

In general, what i am looking for is a solution to have sublime text to work like if i run python inside Powershell (as it looks that everytime i will be coming across a new error). Is there something you can help me with.

Many thanks.

alecxe · Accepted Answer · 2017-12-19T17:45:47.927

This is indeed a Sublime Text specific problem. Sublime Text has this env dictionary that is going to update environment variables before executing Python.

Dictionary of environment variables to be merged with the current process’ before passing them to cmd.

Use this option, for example, to add or modify environment variables without modifying your system’s settings.

Go to Tools -> Build System -> New Build System and set the PYTHONIOENCODING to utf8:

"env": {"PYTHONIOENCODING": "utf8"},

See more at:

sublime and bs4 error (windows 10)

1 Answers1