Socket.io connections distribution between several servers

Question

I'm working on DB design tool (python, gevent-socket.io). In these tool multiple users can discuss one DB model, receiving changes in runtime. To support this feature, I'm using socket.io. I'd like to extend number of servers that handle socket.io connection easily. The simplest way to do it is to set up nginx to choose server basing of model ID.

I'd like module approach, where model ID is divided by number of servers. So if I have 3 nodes, model 1 will be handled on first, 2 - on second, 3 - on third, 4 - on first again etc.

My request for model loading looks like /models/, so no problem here - argument can be parsed to find server to handle it. But after model page is loaded, JS tries to establish connection:

var socket = io.connect('/models', {
            'reconnection limit': 4000
        });

It accesses default endpoint, so server receives following requests:

http://example.com/socket.io/1/xhr-pooling/111111?=1111111

To handle it, I create application this way:

SocketIOServer((app.config['HOST'], app.config['PORT']), app, resource='socket.io', transports=transports).serve_forever()

and then

@bp.route('/<path:remaining>')
def socketio(remaining):
    app = current_app._get_current_object()
    try:
        # Hack: set app instead of request to make it available in the namespace.
        socketio_manage(request.environ, {'/models': ModelsNamespace}, app)
    except:
        app.logger.error("Exception while handling socket.io connection", exc_info=True)
    return Response()

I'd like to change it to

http://example.com/socket.io/<model_id>/1/xhr-pooling/111111?=1111111

to be able to choose right server in ngnix. How to do it?

UPDATE

I also like to check user permissions when it tries to establish connection. I'd like to do it in socketio(remaining) method, but, again, I need to know what model he is trying to access.

UPDATE 2

I implemented permission validator, taking model_id from HTTP_REFERER. Seems, it's only part of request that contains identifier of the model (example of values: http://example.com/models/1/).

You mean different *nginx* server or different *database* server? — gioi, Sep 02 '13 at 09:57
I have an application that handles Socket.io connection. I run several instances of this application and set up nginx to distribute requests evenly between them. So I have several Flask apps and one ngnix before them. — Marboni, Sep 02 '13 at 11:43

Elephant · Accepted Answer · 2013-09-03T08:46:52.103

2

The first idea - is to tell client side available servers for current time. Furthermore you can generate server list for client side by priority, just put them in javascript generated array by order. This answer means that your servers can answer on any models, you can control server loading by changing servers ordering in generated list for new clients.

I think this is more flexible way. But if you want - you can parse query string in nginx and route request on any underlying server - just have a table for "model id-server port" relations

Upd: Just thinking about your task. And find one another solution. When you generate client web page you can inline servers count in js somewhere. Then, when you requesting model updates, just use another parameter founded as

serverId = modelId%ServersCount;

that will be server identificator for routing in nginx. Then in nginx config you can use simple parsing query string, and routing request to server you can find by serverId parameter.

in "metalanguage" it will be

get parameter serverId to var $servPortSuffix
route request to localhost:80$servPortSuffix

or another routing idea.

You can add additional GET parameters to socket.io via

io.connect(url, {query: "foo=bar"})

edited Sep 03 '13 at 08:46

answered Sep 02 '13 at 15:08

Elephant

1,346
10
11

Thanks for your reply. You wrote it right for regular requests, when client requests some page and server returns it. But here I have more complex situation - I use socket.io library and it creates requests instead of me, so I don't have control here. Besides, my nodes are stateful, so I can't process users of the same model on the different servers. – Marboni Sep 02 '13 at 15:41
I already worked with socket.io. You pass url and can pass several parameters to it. You don't need to control something in socket io – Elephant Sep 02 '13 at 15:43
http://stackoverflow.com/questions/13745519/send-custom-data-along-with-handshakedata-in-socket-io look here how to pass aditional params to socket io – Elephant Sep 02 '13 at 15:48
Thanks for link to this post. `io.connect(url, {query: "foo=bar"})` is what I need. Now I can use additional GET parameter to figure out what is the ID of the model and what server should handle it. – Marboni Sep 03 '13 at 07:57
Please, add information about modification of socket.io request to your post and I'll close this question. – Marboni Sep 03 '13 at 07:59

Socket.io connections distribution between several servers

1 Answers1