0

I am trying to get HTML body from Spotify web. But after I output it to the file the result is for some reason different from the original HTML (it's a completely different site).

curl https://open.spotify.com/artist/4npEfmQ6YuiwW1GpUmaq3F > test.html

Eventually, I will do in python so if anyone knows how to get around this page redirect, please help.

Tomáš König
  • 66
  • 1
  • 9

2 Answers2

2

Spotify recognize that you use unsupported "browser", Curl is not a browser so don't think it will behave like one you will need to "fake" that you use a real browser by adding the right headers, something like:

curl 'https://open.spotify.com/artist/4npEfmQ6YuiwW1GpUmaq3F' \
-X 'GET' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15'
Dharman
  • 30,962
  • 25
  • 85
  • 135
LichKing
  • 256
  • 1
  • 2
  • 15
  • And do you think is possible with some lib in python? – Tomáš König Dec 21 '20 at 12:36
  • Yes, you can use Selenium with chrome driver that will act as a real browser here is a helpful question https://stackoverflow.com/questions/42478591/python-selenium-chrome-webdriver – LichKing Dec 21 '20 at 12:45
  • I know is possible with the selenium web drive but that will open the browser ... or it is also possible to do it on background? – Tomáš König Dec 21 '20 at 12:55
  • 1
    Yes. See the solutions proposed [here](https://stackoverflow.com/questions/16180428/can-selenium-webdriver-open-browser-windows-silently-in-the-background). – costaparas Dec 21 '20 at 12:58
-1

First you need to install requests with "pip install requests" in cmd

Next you can use it(you can also add header with user-agent if needed).

import requests
url="google.com" #for example
headers={"User-Agent":"Apple Web Kit"}#adding user agent
html=requests.get(url,headers=headers).content.decode()
Rostcraft
  • 3
  • 7