6

I have been working with pytrends, a package to retrieve google trends data, a long while now and realised that the results I get on the browser and using pytrends differ quite a bit. After checking the request each are doing, the only difference I was able to spot was the parameter user type inside the request they both made, and some minor changes, browser makes the request indicating timezone twice, for example.

Browser:
"userConfig":{"userType":"USER_TYPE_LEGIT_USER"}
Pytrends:
"userConfig": {"userType": "USER_TYPE_SCRAPER"}

Timeframes, timezones, and the rest of parameters in the requests are the same but the token, which you have to get before doing the actual request for the data. I don't know why this may be happening, and I don't feel the token has anything to do, as you request it, again, with the only difference in both request of specifying different user types

Now I post the two full request each make but the token:

Pytrends

https://trends.google.com/trends/api/widgetdata/multiline?req={"time": "2014-12-28 2020-01-01", "resolution": "WEEK", "locale": "es", "comparisonItem": [{"geo": {"region": "ES-CM"}, "complexKeywordsRestriction": {"keyword": [{"type": "BROAD", "value": "gripe"}]}}], "requestOptions": {"property": "", "backend": "IZG", "category": 0}, "userConfig": {"userType": "USER_TYPE_SCRAPER"}}&token=TOKEN_HERE&tz=-120

Browser
https://trends.google.es/trends/api/widgetdata/multiline?hl=es&tz=-120&tz=-120&req={"time":"2014-12-28 2020-01-01","resolution":"WEEK","locale":"es","comparisonItem":[{"geo":{"region":"ES-CM"},"complexKeywordsRestriction":{"keyword":[{"type":"BROAD","value":"gripe"}]}}],"requestOptions":{"property":"","backend":"IZG","category":0},"userConfig":{"userType":"USER_TYPE_LEGIT_USER"}}&token=TOKEN_HERE

Does anyone know why is this happening and how can I manage to retrieve consistent data between both of them? Also, why does Google give different data if you are scraping their website rather than using a browser?

Apollo
  • 164
  • 8
  • Google may remember all your visted pages in browser and use this to send different content. – furas Oct 07 '22 at 14:52
  • Not quite, results are the same even in a different computer with a different account. Only change produced when it detects data is being scraped. – Apollo Oct 07 '22 at 17:07
  • Technically speaking, pytrends is a scraper, so the the userType does seems correct. – Shalev Manor May 28 '23 at 13:50
  • Yeah, but that's the thing, not that it shouldn't be detected as a scraper, but that it retrieves different info for scrapers. Why? Which info is correct? Are both just different samples of the total of searches? – Apollo Jun 24 '23 at 17:25

0 Answers0