2

I'm fairly new to scripting in general and I'm pretty sure this is trivial but i can't seem to find a solution. I want to use the python websockets library to listen to multiple websockets in order to get ticker information about crypto prices.

How to get real time bid / ask / price from GDAX websocket feed provides a good start for obtaining the feed for one currency.

The problem is that the run_forever() does not allow me to show two feeds at the same time as i have no way to interrupt it.

Asplund
  • 33
  • 1
  • 6

1 Answers1

3

The GDAX websocket allows you to subscribe to multiple pairs. As seen below I subscribe to both the BTC-USD and ETH-USD pairs. I assume you can subscribe to unlimited pairs.

import websocket
from json import dumps, loads
try:
    import thread
except ImportError:
    import _thread as thread

def on_message(ws, message):
    parsed_msg = loads(message)
    print(parsed_msg["product_id"], parsed_msg["price"])

def on_open(ws):
    def run(*args):
        params = {
            "type": "subscribe",
            "channels": [{"name": "ticker", "product_ids": ["BTC-USD", "ETH-USD"]}]
        }
        ws.send(dumps(params))
    thread.start_new_thread(run, ())

if __name__ == "__main__":
    websocket.enableTrace(True)
    ws = websocket.WebSocketApp("wss://ws-feed.gdax.com", on_open=on_open, on_message = on_message)
    ws.run_forever()

If for some reason GDAX did not allow this, you could open multiple web sockets in multiple threads, but in this case its not necessary.

Tom
  • 1,636
  • 2
  • 13
  • 21
  • 1
    upvoted! so just a quick question if you dont mind answering, the above solution works really well for say GDAX, but if you had multiple exchanges, would you run them each inside a different Thread? if by any chance you are familiar with node, how would you do it there given we got only one process with an event loop running – PirateApp Sep 03 '18 at 13:22
  • 1
    You could, but a better solution, especially if you find yourself dealing with many connections would be to use an event loop with asyncio. As you said Nodejs uses an event loop by default so its even easier. You can use this library to work with websockets: https://github.com/websockets/ws. Event loops are intended to handle 1000's of requests with a single thread. Yes if two open sockets receive data at exactly the same instant, they will not be processed in parallel. Once will happen before the other, but this is intentional. I could go into more detail but I've run out of characters lol. – Tom Sep 03 '18 at 14:18
  • thank you so much! my concern was if you had say 10 connections to different exchanges and they all stream 100000 items every second or so, would asyncio also become a bottleneck at some point, so if asyncio could be run in a multithreaded manner or if node websocket or socketio library could be run over several processes i think it could be a decent solution for a considerable amount of time, I coudnt find any info on whether this was possible, hence the comment :) – PirateApp Sep 03 '18 at 15:25
  • Sorry just found the answer for Python , I think this should do the trick https://gist.github.com/lars-tiede/01e5f5a551f29a5f300e though I have no idea how to replicate this in node.js I see a lot of examples of websocket server on node js using cluster but nothing for the client side, wonder if this worker.js file can be modified to support a client instead of server, wait it will simply ping the same url 8 times if I have 8 CPUs, wont it https://github.com/Roger13/multiprocess-websocket-server/tree/master/lib – PirateApp Sep 03 '18 at 15:33
  • 1
    Yeah when people use cluster, they are forking the main node process. So for 8 cores you will have 8 identical processes handling requests which should roughly increase the amount of requests/per second the server can handle by x8. But for your case, it wont work, although its probably technically possible to have each process run different code and connect to different sockets, you definately shouldn't do that, sounds like a big anti-pattern. – Tom Sep 04 '18 at 04:17
  • 1
    You should just be able to open multiple socket connections in the same thread, I think it should be fast enough, as long as your not doing any slow blocking computations in your callback functions like encryption, synchronous file I/O etc. If you could try it and get back to me I'd be interested to see how it goes as I haven't personally done something like this before. Also maybe we should take this to chat? – Tom Sep 04 '18 at 04:18
  • thanks! sure we can take this over chat, I found an implementation of the scaled client and am gonna test this https://github.com/Roger13/multiprocess-websocket-server/tree/multiprocess-websocket-client at the moment, will get back on chat with the results, this is rather new even for me – PirateApp Sep 04 '18 at 04:30