Recieve JSON data from link in whattomine without scraping HTML

Question

Explanation

This link is where you are sent to after entering in your hardware stats (hashrate, power, power cost, etc.). On the top bar (below the blue Twitter follow button) is a link to a JSON file created after the page loads with the hardware stats information entered; clicking on that JSON link redirects you to another URL (https://whattomine.com/asic.json).

Goal

My goal is to access that JSON file directly after manipulating the values in the URL string via the terminal. For example, if I would like to change hashrate from 100 to 150 in this portion of the URL:

[sha256_hr]=100& ---> [sha256_hr]=150&

After the URL manipulations (like above, but not limited to), I would like to receive the JSON output so that I can pick-out the desired data.

My Code

Advisory: I started Python programming ~June 2017, please forgive.

import json
import pandas as pd
import urllib2
import requests


hashrate_ghs = float(raw_input('Hash Rate (TH/s): '))
power_W = float(raw_input('Power of Miner (W): '))
electric_cost = float(raw_input('Cost of Power ($/kWh): '))
hashrate_ths = hashrate_ghs * 1000

initial_request = ('https://whattomine.com/asic?utf8=%E2%9C%93&sha256f=true&factor[sha256_hr]={0}&factor[sha256_p]={1}&factor[cost]={2}&sort=Profitability24&volume=0&revenue=24h&factor[exchanges][]=&factor[exchanges][]=bittrex&dataset=Main&commit=Calculate'.format(hashrate_ths, power_W, electric_cost))
data_stream_mine = urllib2.Request(initial_request)

json_data = requests.get('https://whattomine.com/asic.json')
print json_data

Error from My Code

I am getting an HTTPS handshake error. This is where my Python freshness is second most blatantly visible:

Traceback (most recent call last):
  File "calc_1.py", line 16, in <module>
    s.get('https://whattomine.com/asic.json')
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='whattomine.com', port=443): Max retries exceeded with url: /asic.json (Caused by SSLError(SSLError(1, u'[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)'),))

Thank you for your help and time!

Please advise me of any changes or for more information concerning this question.

While for some it seemed to be like a [pyOpenSSL version issue](https://github.com/requests/requests/issues/4246#issuecomment-323398451), uninstalling and reinstalling which has fixed the problem. Another older answer in SO asks to [do the following](https://stackoverflow.com/questions/24422096/python-requests-and-openssl-httpsconnectionpool-max-retries-exceeded) — Sudheesh Singanamalla, Nov 13 '17 at 07:04
Much love @SudheeshSinganamalla. If you would like to post that as an answer, I will be more than happy to upvote it. Your suggestion of updating pyOpenSSL worked for me, I now receive `` Thank you! — 86_951, Nov 13 '17 at 19:20
No joy yet @garej, I still can't seem to get past just the default JSON data; as in, the URL manipulations are not registered in the JSON results. The answer helped me get a step further. I'm still scratching my head with the hopes someone can help me get another step closer, or some time and experience on my end for that extra step. — 86_951, Dec 20 '17 at 10:57

score 1 · Answer 1 · answered Nov 13 '17 at 19:55

1

It looks like a few others faced similar problems.

While for some it seemed to be like a pyOpenSSL version issue, uninstalling and reinstalling which has fixed the problem. Another older answer in SO asks to do the following.

answered Nov 13 '17 at 19:55

Sudheesh Singanamalla

2,283
3
19
36

score 1 · Accepted Answer · answered Dec 20 '17 at 11:08

This is just a comment. The following approach would suffice (Python 3).

import requests

initial_request = 'http://whattomine.com/asic.json?utf8=1&dataset=Main&commit=Calculate'

json_data = requests.get(initial_request)
print(json_data.json())

The key point here this part - put .json in your initial_request and it will be enough. You may add all you parameters as you did in the query part after ? sign

Recieve JSON data from link in whattomine without scraping HTML

Explanation

Goal

My Code

Error from My Code

2 Answers2