29

I am trying to make web request to my trading account . Python is unable to decode the web request. Web request is successful with code - 200.

Here is the code below

import requests

headers = {
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.9',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36',
    'x-kite-version': '1.2.1',
    'accept': 'application/json, text/plain, */*',
    'referer': 'https://kite.zerodha.com/orders',
    'authority': 'kite.zerodha.com',
    'cookie': '__cfduid=db8fb54c76c53442fb672dee32ed58aeb1521962031; _ga=GA1.2.1516103745.1522000590; _gid=GA1.2.581693731.1522462921; kfsession=CfawFIZq2T6SghlCd8FZegqFjNIKCYuO; public_token=7FyfBbbxhiRRUso3425TViK2VmVszMCK; user_id=XE4670',
    'x-csrftoken': '7FyfBbbxhiRRUso3425TViK2VmVszMCK',
}

response = requests.get('https://kite.zerodha.com/api/orders', headers=headers)


x=str(response.content.decode("utf-8") )

b"1X\x14\x00 \xfe\xa7\x9b\xd3\xca\xbd9-\x12\x83\xbfULS1\x1d8\x9d\x0e\xd4\xcf\xbd\xb8\xd1\xbd4\xc0\x00\x13~\x94}\xe4\x81\xa4\x90P\x1cfs\xcd\x1e\xaeG\x9b},m\xbd\t\x84L1\xde\xa8e\x8a\xf1h\x0e\x0c)\x1a\x12\xfb\x06z\xec\x18\xe4r\xa1\x1c\x11\xe8 \xbcO\xec\xe2|\xa6\x90\xa9\xdf\xf2\xe1\xfa\xf3\x1e\x04\x0e\xa2\x8d\x0e\xc4\tw\xeb\xd9\xba\n\xf1H'l\xeb>\x08\x85L\r\x0cY\xf8\x81D;\x92!o\xfd\xbd\xe3u>3\x10\xe1\x8c;\xb8\x9e\xceA\xae\x0exX\xc9\x19s\xeb\xe5r~1\x98\xed0\xb8\xdc\xb4\x17:\x14\x96xAn\xb9\xf0\xce\xf2l\\xa6G?5O\x9b\xf3\xc1\\x1f\x0f\x8fs\x1b/\x17\x1a\x0c[ySAX\x1d'\xe7\xbb\nx\xacR~\xbb\x9f\xe0\x8c?s\xc0\x8f\xe0\x97\xff\xde'\xc7#\x8f\x97\xaf\xaa%\xf2\xf9\xfaC|\xcf\t\xf3\xeb\xaa\xdcs\xcc\xf5\xa3RM\xbaOY\xf5\x9fe\xfc\x07\xff\x01"

Unable to decode this. Tried unicode- utf 8 and various codes available on the stakoverflow but its failed.

DYZ
  • 55,249
  • 10
  • 64
  • 93
dnyanesh ambhore
  • 441
  • 1
  • 4
  • 8

3 Answers3

38

According to the response.headers (that you did not provide, but that are easily recoverable by running your code), the response is encoded using Brotli compression (Content-Encoding': 'br'). You can decompress it with brotlipy:

import brotli
brotli.decompress(response.content)
#b'{"status":"success","data":[{"placed_by":"XE4670","order_id":"180331000000385",  
#"exchange_order_id":null,"parent_order_id":null,"status":"REJECTED",
#"status_message":"ADAPTER is down","order_timestamp":"2018-03-31 07:59:42", 
#"exchange_update_timestamp":null,...}

Now, it's JSON, as promised ('Content-Type': 'application/json').

DYZ
  • 55,249
  • 10
  • 64
  • 93
  • Sorry to dig up this post, but when I use `brotly.decompress(response.content)`, I have `brotli.brotli.Error: Decompression error: b'PADDING_2'`, what am I doing wrong ? – robinvrd Dec 23 '19 at 13:57
  • 2
    Use [brotli](https://github.com/google/brotli) instead of `brotlipy`. It is more frequently updated and maintained by google. – Dipu Feb 17 '20 at 11:17
  • sorry, i have a question use brotli.decompress(response.content) raise error https://stackoverflow.com/questions/63809144/python3-8-brotli-brotli-error-brotlidecompress-failed – xin.chen Sep 09 '20 at 10:11
  • 1
    as mentioned in my answer on the above question, once brotli is installed, it is handled automatically by requests. You don't need to call decompresss manually – Laurent Pinson Jul 05 '22 at 14:39
14

If the server only returns brotli compressed response, response should be decompressed then it will be ready to use.

Fortunately, since v2.26.0 update, requests library supports Brotli compression if either the brotli or brotlicffi package is installed. So, if the response encoding is br, requests library will automatically handle it and decompress it.

First;

pip install brotli,

then;

import requests

r = requests.get('some_url')
r.json()
Cem Önel
  • 721
  • 6
  • 8
4

I fixed this issue by changing the request header,

headers = {
    'accept-encoding': 'gzip, deflate, br',
    ...}

to

headers = {
    'accept-encoding': 'gzip, deflate, utf-8',
    ...}
basic mojo
  • 128
  • 2
  • 12