For an small Django (1.10.3) app I'd like to have websockets and I'm new to the whole websockets business. So I searched for a relatively easy to understand project for django that enables websocket connections and I've found django-websocket-redis, which looks easy to use and deploy.
So I created a very small example project (see the github page) where I can test the things I've learnt while reading the documentation of django-websocket-redis and I was very quickly able to write something that works.
If I start the app with ./manage.py runserver 0.0.0.0:3000 and go to http://localhost:3000, then the page is able to make the websocket connection, subscribe to a channel and add a <li>content</li> on the page whenever a new message from the server arrives. In the django app I started a thread that broadcast over the websocket interface randomly between 2 and 9 seconds a random string. I was happy with that and I planned to see if the deployment is as easy as it seems.
So I took a closer look at documentation and almost made copy & paste changing only the things that apply to me. I created a file websockets/wsgi_websocket.py with
import os
import gevent.socket
import redis.connection
redis.connection.socket = gevent.socket
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "websockets.settings")
from ws4redis.uwsgi_runserver import uWSGIWebsocketServer
application = uWSGIWebsocketServer()
and a file websockets/deploy_settings.py with
DEBUG = False
ALLOWED_HOSTS = ['localhost', '127.0.0.1', '::1', '192.168.2.117', 'weby.com']
...
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'ws4redis',
'websockets',
]
....
WSGI_APPLICATION = 'ws4redis.django_runserver.application'
STATIC_URL = '/static/'
STATICFILES_DIRS = [
os.path.join(os.path.dirname(ws4redis.__file__), "static"),
]
STATIC_ROOT = os.path.join(BASE_DIR, "deployment", "static")
WEBSOCKET_URL = '/ws/'
WS4REDIS_EXPIRE = 5
WS4REDIS_PREFIX = 'ws'
SESSION_ENGINE = 'redis_sessions.session'
SESSION_REDIS_PREFIX = 'session'
...
In the index.html page I do the connection as follows:
var ws_url = 'ws://' + window.location.host + '/ws/foobar?subscribe-broadcast&publish-broadcast';
var ws = new WebSocket(ws_url);
...
Like I said, in DEBUG = True mode everything works fine. I installed nginx (I use Gentoo Linux, so I did emerge nginx with these flags enabled: aio http http-cache http2 ipv6 pcre ssl NGINX_MODULES_HTTP=access addition auth_basic autoindex browser charset empty_gif fastcgi geo gzip limit_conn limit_req map memcached proxy referer rewrite scgi split_clients ssi upstream_ip_hash userid uwsgi)
I added this to the default config
upstream django_upstream {
server 127.0.0.1:17999;
}
upstream websocket_upstream {
server 127.0.0.1:18000;
}
server {
listen 0.0.0.0:80;
server_name weby.com;
access_log /tmp/web.com-access.log;
error_log /tmp/web.com-error.log;
location /static {
alias /home/shaoran/projects/python/websockets/deployment/static;
}
location / {
uwsgi_pass django_upstream;
include /etc/nginx/uwsgi_params;
}
location /ws/ {
proxy_pass http://websocket_upstream;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
I added weby.com to my /etc/hosts file and resolves to 127.0.0.1. The last line in the nginx config I added after looking for my problem and found this (see tylercbs comment), so I added theproxy_set_header Host $host;` line.
The uWSGI instances were started like this:
# exporting DJANGO_SETTINGS_MODULE
export DJANGO_SETTINGS_MODULE=websockets.deploy_settings
# the main django app
uwsgi --socket 127.0.0.1:17999 --wsgi-file websockets/wsgi.py --die-on-term --buffer-size=32768 --workers=5 --master --logto2 /tmp/wsgi_django.log
# the websocket app
uwsgi --socket 127.0.0.1:18000 --wsgi-file websockets/wsgi_websocket.py --die-on-term --gevent 1000 --http-websockets --workers=2 --master --logto2 /tmp/wsgi_websocket.log
Now when I go to http://weby.com the page doesn't show me the random strings. I opened the console (testing with google chrome) and I saw this error:
WebSocket connection to 'ws://weby.com/ws/foobar?subscribe-broadcast&publish-broadcast' failed: Error during WebSocket handshake: Unexpected response code: 502
In the console I executed again
ws = new WebSocket("ws://weby.com/ws/foobar?subscribe-broadcast&publish-broadcast")
to see what error message I would get in the log files. The django log files had nothing. The nginx error log file says
2016/11/21 01:16:17 [error] 12892#0: *1 upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1, server: weby.com, request: "GET /ws/foobar?subscribe-broadcast&publish-broadcast HTTP/1.1", upstream: "http://127.0.0.1:18000/ws/foobar?subscribe-broadcast&publish-broadcast", host: "weby.com"
The uWSGI log file says
invalid request block size: 21573 (max 4096)...skip
So I googled that and found this: uwsgi invalid request block size, so I added the -b 32768 option to the uwsgi command for the websocket interface. When I executed
ws = new WebSocket("ws://weby.com/ws/foobar?subscribe-broadcast&publish-broadcast")
I didn't get any error message again. "Hooray" I thought, but no message arrived. After a couple of seconds I got the Error during WebSocket handshake: Unexpected response code: 502 again :(
So I added the --protocol=http as suggested on the same thread and I got one message but nothing more. The uWSGI log says only
[pid: 13513|app: 0|req: 1/1] 127.0.0.1 () {42 vars in 786 bytes} [Mon Nov 21 00:42:08 2016] GET /ws/foobar?subscribe-broadcast&publish-broadcast => generated 58 bytes in 64418 msecs (HTTP/1.1 101) 4 headers in 178 bytes (3 switches on core 999)
and the django log says
Subscribed to channels: subscribe-broadcast, publish-broadcast
I reloaded the page and I finally got one message but after a couple of seconds the messages were not coming. I reloaded the page again with F5 and nothing, not even the first message like before.
So I tried closing the connection with ws.close() I got
WebSocket connection to 'ws://weby.com/ws/foobar?subscribe-broadcast&publish-broadcast' failed: One or more reserved bits are on: reserved1 = 1, reserved2 = 0, reserved3 = 1
Websocket connection is broken!
and the django log says
WebSocketError: unable to receive websocket message
Traceback (most recent call last):
File "/home/shaoran/anaconda/websockets/lib/python2.7/site-packages/ws4redis/wsgi_server.py", line 120, in __call__
recvmsg = RedisMessage(websocket.receive())
File "/home/shaoran/anaconda/websockets/lib/python2.7/site-packages/ws4redis/uwsgi_runserver.py", line 31, in receive
raise WebSocketError(e)
WebSocketError: unable to receive websocket message
I feel that I'm getting closer to the correct configuration, but here I don't know what goes wrong. The nginx config seems to be alright, so it must be something on how I execute uwsgi.
The thing is that is seems to work only randomly. I reloaded the page but this time I didn't do anything just kept waiting and see if something appears. I opened firefox and loaded the page, to see if this was a chrome issue. With firefox I had exactly the same, the only difference is that firefox doesn't show any error message when closing the connection, but the djangolog file shows the WebSocketError exception. I returned to the chrome page I reloaded a couple of minutes ago and to my surprise I found 5 new messages. I cannot reproduce this behaviour, it is always a different outcome.
What am I doing wrong?
edit
I think I found the problem. I modified my thread so:
# inside websockets/views.py
class WS(object):
def __init__(self):
self.counter = 0
def listen_and_replay_to_redis(self):
logger = logging.getLogger("django")
logger.debug(" >> websocket starting thread")
try:
redis_publisher = RedisPublisher(facility='foobar', broadcast=True)
while True:
data = str(uuid.uuid4())
#self.counter += 1
#data = "%s - %s" % (data, self.counter)
redis_publisher.publish_message(RedisMessage(data))
ttw = random.uniform(3, 10)
ttw = 1
logger.debug(" >> websocket thread %s: %s waiting %s seconds" % (datetime.now().strftime("%H:%M:%S"), data, ttw))
time.sleep(ttw)
except Exception as e:
logger.debug(" >> websocket thread error: %s" % e)
logger.debug(" >> websocket thread dying")
obj = WS()
th = threading.Thread(target = obj.listen_and_replay_to_redis)
th.start()
When I start in debug mode I can see in the log file that the thread is running because it generates messages like this:
>> websocket thread 02:09:15: a2d460d4-a660-47ad-845d-1a60a6dddb14 waiting 1 seconds
>> websocket thread 02:09:16: 9313a0e9-8d07-42a7-8cd5-a2d1951d8ba0 waiting 1 seconds
>> websocket thread 02:09:17: fbbfc063-ecb2-4424-a017-1f943d3fdc2d waiting 1 seconds
When I start in production mode with wsgi I don't see anything. The first I make a request (curl http://weby.com) the log file shows
>> websocket starting thread
and nothing more. If I keep doing curl http://weby.com I sometimes see "websocket starting thread" message, sometimes the other lines. So that shows that either RedisPublisher and redis_publisher.publish_message block or the thread is killed. I've been reading different threads here about launching threads, I implemented the suggestions like starting the thread in the urls.py module (as it get loaded only once) or in the uwsgi.py file itself, nothing seems to work. How can I have a thread running in the background when deploying with uWSGI?
