Post request won't return appropriate data

Question

I tried to scrape some data from a national betting site called https://tippmix.hu. I wanted to get the data from this specific page:https://www.tippmix.hu/sportfogadas#?sportid=999&countryid=99999999&competitionid=45975&page=1. The data is dynamically loaded so I inspected the page and found a specific POST request which is responsible for loading the data I need in json. This is it.

I opened python and used the requests library to make a post request to this page: https://api.tippmix.hu/tippmix/search with exact same headers and data you can see on my previous picture. Unfortunately, my code returned the whole json file as if I did not specified any parameters. Here is my code (the header_convert function converts the copied header string into a dictionary:

import requests
from header_convert import header_convert

events_url = "https://api.tippmix.hu/tippmix/search"
data = {"fieldValue": "",
        "sportId": "999",
        "competitionGroupId": "99999999",
        "competitionId": "45975",
        "type": "0",
        "date": "0001-01-01T00:00:00.000Z",
        "hitsPerPage": "20",
        "page": "1",
        "minOdds": "null",
        "maxOdds": "null"}

raw_headers = """Accept: application/json, text/plain, */*
Content-Type: application/x-www-form-urlencoded
Origin: https://www.tippmix.hu
Content-Length: 182
Accept-Language: en-us
Host: api.tippmix.hu
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15
Referer: https://www.tippmix.hu/
Accept-Encoding: gzip, deflate, br
Connection: keep-alive"""
headers = header_convert.header_convert(raw_headers)
print(headers)

page = requests.post(events_url, data=data, headers=headers)
print(page.content)

Here are my headers:

{'Accept': 'application/json, text/plain, */*', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'https://www.tippmix.hu', 'Content-Length': '182', 'Accept-Language': 'en-us', 'Host': 'api.tippmix.hu', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15', 'Referer': 'https://www.tippmix.hu/', 'Accept-Encoding': 'gzip, deflate, br', 'Connection': 'keep-alive'}

I wonder if someone could help me. Thank you!

childnick · Accepted Answer · 2022-02-03T22:51:49.930

0

You need to post the data as json, updated with working code:

import requests
import json

url = 'https://api.tippmix.hu/tippmix/search'

data = {"fieldValue":"",
    "sportId":999,
    "competitionGroupId":99999999,
    "competitionId":45975,
    "type":0,
    "date":"0001-01-01T00:00:00.000Z",
    "hitsPerPage":20,
    "page":1,
    "minOdds":"null",
    "maxOdds":"null"}

headers = {
    'Accept':'application/json, text/plain, */*',
    'Content-Type':'application/x-www-form-urlencoded',
    'Host':'api.tippmix.hu',
    'Origin':'https://www.tippmix.hu',
    'Referer':'https://www.tippmix.hu/',
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36'
    }

resp = requests.post(url,headers=headers,data=json.dumps(data)).json()
print(resp)

edited Feb 03 '22 at 22:51

answered Feb 03 '22 at 16:53

childnick

1,361
1
6
10

As far as I know and the documentation also say it, it's fine to use python dictionaries as parameters. https://docs.python-requests.org/en/latest/user/quickstart/ – vate Feb 03 '22 at 17:55
Ok, your code didn't work... then I made that change and it worked? – childnick Feb 03 '22 at 20:09
I tried it and it still doesn't give back the wanted results. – vate Feb 03 '22 at 20:50
1

see my code in my original comment – childnick Feb 03 '22 at 22:52
You are right, I am sorry. It just confuses my why won't it work with python dictionaries. – vate Feb 04 '22 at 10:52
I found an answer if you are interested: https://stackoverflow.com/a/15696705/13964251 – vate Feb 04 '22 at 10:54
I think it is because my 'data' has ints and yours were strings – childnick Feb 04 '22 at 11:14
Yeah, probably. Thank you anyways! – vate Feb 04 '22 at 11:36

Post request won't return appropriate data

1 Answers1