I want to do my own bitcoin chart.
Would you know any reliable way to retrieve bitcoins historical price data? Is there any way to retrieve it using REST? I saw Bitfloor, which supports REST, but it is not returning any useful value, it has a "internal server error".
I saw also Bitcoincharts, but I think it's limited to 2000 data values.
Would you suggest me any framework or system to work about it?
Actually, you CAN get the whole Bitcoin trades history from Bitcoincharts in CSV format here : http://api.bitcoincharts.com/v1/csv/
it is updated twice a day for active exchanges, and there is a few dead exchanges, too.
EDIT: Since there are no column headers in the CSVs, here's what they are : column 1) the trade's timestamp, column 2) the price, column 3) the volume of the trade
You can find a lot of historical data here: https://www.quandl.com/data/BCHARTS-Bitcoin-Charts-Exchange-Rate-Data
In case, you would like to collect bitstamp trade data form their websocket in higher resolution over longer time period you could use script log_bitstamp_trades.py below.
The script uses python websocket-client and pusher_client_python libraries, so install them.
#!/usr/bin/python
import pusherclient
import time
import logging
import sys
import datetime
import signal
import os
logging.basicConfig()
log_file_fd = None
def sigint_and_sigterm_handler(signal, frame):
global log_file_fd
log_file_fd.close()
sys.exit(0)
class BitstampLogger:
def __init__(self, log_file_path, log_file_reload_path, pusher_key, channel, event):
self.channel = channel
self.event = event
self.log_file_fd = open(log_file_path, "a")
self.log_file_reload_path = log_file_reload_path
self.pusher = pusherclient.Pusher(pusher_key)
self.pusher.connection.logger.setLevel(logging.WARNING)
self.pusher.connection.bind('pusher:connection_established', self.connect_handler)
self.pusher.connect()
def callback(self, data):
utc_timestamp = time.mktime(datetime.datetime.utcnow().timetuple())
line = str(utc_timestamp) + " " + data + "\n"
if os.path.exists(self.log_file_reload_path):
os.remove(self.log_file_reload_path)
self.log_file_fd.close()
self.log_file_fd = open(log_file_path, "a")
self.log_file_fd.write(line)
def connect_handler(self, data):
channel = self.pusher.subscribe(self.channel)
channel.bind(self.event, self.callback)
def main(log_file_path, log_file_reload_path):
global log_file_fd
bitstamp_logger = BitstampLogger(
log_file_path,
log_file_reload_path,
"de504dc5763aeef9ff52",
"live_trades",
"trade")
log_file_fd = bitstamp_logger.log_file_fd
signal.signal(signal.SIGINT, sigint_and_sigterm_handler)
signal.signal(signal.SIGTERM, sigint_and_sigterm_handler)
while True:
time.sleep(1)
if __name__ == '__main__':
log_file_path = sys.argv[1]
log_file_reload_path = sys.argv[2]
main(log_file_path, log_file_reload_path
and logrotate file config
/mnt/data/bitstamp_logs/bitstamp-trade.log
{
rotate 10000000000
minsize 10M
copytruncate
missingok
compress
postrotate
touch /mnt/data/bitstamp_logs/reload_log > /dev/null
endscript
}
then you can run it on background
nohup ./log_bitstamp_trades.py /mnt/data/bitstamp_logs/bitstamp-trade.log /mnt/data/bitstamp_logs/reload_log &
Bitstamp has live bitcoin data that are publicly available in JSON at this link. Do not try to access it more than 600 times in ten minutes or else they'll block your IP (plus, it's unnecessary anyway; read more here). The below is a C# approach to getting live data:
using (var WebClient = new System.Net.WebClient())
{
var json = WebClient.DownloadString("https://www.bitstamp.net/api/ticker/");
string value = Convert.ToString(json);
// Parse/use from here
}
From here, you can parse the JSON and store it in a database (or with MongoDB insert it directly) and then access it.
For historic data (depending on the database - if that's how you approach it), do an insert from a flat file, which most databases allow you to use (for instance, with SQL Server you can do a BULK INSERT from a CSV file).
I have written a java example for this case:
Use json.org library to retrieve JSONObjects and JSONArrays. The example below uses blockchain.info's data which can be obtained as JSONObject.
public class main
{
public static void main(String[] args) throws MalformedURLException, IOException
{
JSONObject data = getJSONfromURL("https://blockchain.info/charts/market-price?format=json");
JSONArray data_array = data.getJSONArray("values");
for (int i = 0; i < data_array.length(); i++)
{
JSONObject price_point = data_array.getJSONObject(i);
// Unix time
int x = price_point.getInt("x");
// Bitcoin price at that time
double y = price_point.getDouble("y");
// Do something with x and y.
}
}
public static JSONObject getJSONfromURL(String URL)
{
try
{
URLConnection uc;
URL url = new URL(URL);
uc = url.openConnection();
uc.setConnectTimeout(10000);
uc.addRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)");
uc.connect();
BufferedReader rd = new BufferedReader(
new InputStreamReader(uc.getInputStream(),
Charset.forName("UTF-8")));
StringBuilder sb = new StringBuilder();
int cp;
while ((cp = rd.read()) != -1)
{
sb.append((char)cp);
}
String jsonText = (sb.toString());
return new JSONObject(jsonText.toString());
} catch (IOException ex)
{
return null;
}
}
}
Coinbase has a REST API that gives you access to historical prices from their website. The data seems to show the Coinbase spot price (in USD) about every ten minutes.
Results are returned in CSV format. You must query the page number you want through the API. There are 1000 results (or price points) per page. That's about 7 days' worth of data per page.
Scraping it to JSON with Node.js would be fun :)
https://github.com/f1lt3r/bitcoin-scraper
[
[
1419033600, // Timestamp (1 for each minute of entire history)
318.58, // Open
318.58, // High
318.58, // Low
318.58, // Close
0.01719605, // Volume (BTC)
5.478317609, // Volume (Currency)
318.58 // Weighted Price (USD)
]
]
