3

I am using Python Requests to access an HTTPS page that contains JSON data. I pass 2 parameters in the get request, they are listed below verify = False cookies = cookie I am aware that the verify = False is to get past SSL certification verification, and the cookies = cookie parameter is to pass cookie values. There are actually 2 cookie values that I have put in a dictionary, and one of the values is very long maybe 300+ characters. But when I run my code I get this error

UnicodeEncodeError: 'latin-1' codec can't encode character '\u2026' in position 602: ordinal not in range(256)

My code is attached below

import requests
test_url = "https://abc/cde"
cookie = {"cookie1":"value1(10 characters)", "cookie2":"value2(300+ characters)"}
response = requests.get(test_URL, verify=False, cookies= cookie)
print(response.content)
PSR
  • 249
  • 1
  • 10

1 Answers1

3

\u2026 is the HORIZONTAL ELLISPSIS character which is not a legal character in cookies (see the answers to this Q&A

>>> r = requests.get('https://www.google.com', cookies={'mycookie': '\u2026'})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/me/virtual-envs/so310/lib/python3.10/site-packages/requests/api.py", line 75, in get 
    return request('get', url, params=params, **kwargs)
  File "/home/me/virtual-envs/so310/lib/python3.10/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
...
  File "/usr/local/lib/python3.10/http/client.py", line 1317, in _send_request
    self.putheader(hdr, value)
  File "/home/me/virtual-envs/so310/lib/python3.10/site-packages/urllib3/connection.py", line 224, in putheader
    _HTTPConnection.putheader(self, header, *values)
  File "/usr/local/lib/python3.10/http/client.py", line 1249, in putheader
    values[i] = one_value.encode('latin-1')
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2026' in position 9: ordinal not in range(256)

The usual solution to this problem is to encode the value with a method that doesn't expose illegal characters, for example base64, or JSON with unicode characters escaped, assuming that the consumer of the cookies will know how to decode the value. Otherwise, remove such characters from the cookie value.

snakecharmerb
  • 47,570
  • 11
  • 100
  • 153
  • Hey, i have just 2 instances of that particular character \u2026 .... its somewhere in the middle of the cookie value, is it fine if i delete just that? will it not affect me beeing able to perform requests ? @snakecharmerb – PSR Jan 07 '22 at 11:24
  • 1
    That should be fine, assuming it doesn't affect how the cookie is interpreted by whatver systems are consuming it. – snakecharmerb Jan 07 '22 at 12:23