0

I want to set a date range on the web page:

https://www.investing.com/crypto/bitcoin/btc-usd-historical-data?cid=49799

Greatful for any help!

I tried using data payload using 'picker' value but it didnt work

ticker = [x.strip() for x in 
    open("F:\\System\\PVWAVE\\Crypto\\tickers.txt", "r").readlines()]
urlheader = {
  "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36",
  "X-Requested-With": "XMLHttpRequest"
}
payload = {'picker' : ' 06/01/2016 - 12/22/2018'}    
url = "https://www.investing.com/crypto/"+ticker+"-historical-data?cid=997651"
req = requests.get(url, headers=urlheader, data=payload)

Date range needs to be set to 1st June 2016 to yesterday (every time I run it)

  • 1
    From what I can tell, the data is being updated / retrieved in the picker via Ajax. You would need to [inspect this Ajax call in the network tab of your browser](https://stackoverflow.com/questions/1820927/request-monitoring-in-chrome. However, if you are looking to get a straight feed of BTC to USD, there are [many better options than scraping](https://www.coindesk.com/api). – Nikolaj Baer Dec 22 '18 at 20:33
  • I just need a once daily download across many pairs and many exchanges. Ajax does bring back the data (no idea what Ajax is!) and I have downloaded the .HAR file. It seems to contain some fields for POST. Can I just specify those in the requests.get? –  Dec 22 '18 at 21:07
  • It seems your question is related to another question which has already been answered. Here is the link for your question - https://stackoverflow.com/questions/53890493/scraping-data-from-investing-com-for-btc-eth-using-beautifulsoup – Siddhartha Dec 29 '18 at 03:52

0 Answers0