Passing a non ascii character in steam url

Question

I'm trying to fetch the price of an item StatTrak™ Dual Berettas | Panther (Factory New). however the TM is causing issue as urllib treats it as a non-ascii character. I've found this How to fetch a non-ascii url with urlopen? but for whatever reason steam is passing a 500 error

from urllib.parse import quote 

def get_price(item_name):
    base_url = 'http://steamcommunity.com/market/priceoverview/?appid=730&currency=1&market_hash_name={}'
    print(base_url.format(quote(item_name)))
    request = urllib.request.urlopen(base_url.format(quote(item_name)))

Output URL (after parse): http://steamcommunity.com/market/priceoverview/?appid=730&currency=1&market_hash_name=StatTrak%E2%84%A2%2520Dual%2520Berettas%2520%7C%2520Panther%2520%28Factory%2520New%29

Working url (done in browser): https://steamcommunity.com/market/priceoverview/?appid=730&currency=1&market_hash_name=StatTrak%E2%84%A2%20Dual%20Berettas%20|%20Panther%20(Factory%20New)

I seem to need to pass this, but urllib won't allow me to. What could I do?

Answer 1

You code seems to work fine:

>>> base_url = 'http://steamcommunity.com/market/priceoverview/?appid=730&currency=1&market_hash_name={}'
>>> urllib.request.urlopen(base_url.format(quote("StatTrak™ Dual Berettas | Panther (Factory New)"))).read()
b'{"success":true,"lowest_price":"$5.80","volume":"3","median_price":"$4.53"}'

I believe your issue arises because of double quoting. The %2520 in your "Output URL" means you quoted a single space (%20) twice (' ' -> %20 -> %2520).

Your code however seems to quote only once, which is good. You must have passed the item already quoted to the function.