4

I have Django application with celery, which works fine.

app = Celery('backend', broker='amqp://guest:guest@localhost:5672//',)

Then I tried to cipher connection with SSL:

app = Celery('backend', broker='amqp://guest:guest@localhost:5672//',)
app.config_from_object('django.conf:settings', namespace='CELERY')

And with settings.py:

import ssl

CELERY_BROKER_USE_SSL = {
  'keyfile': '/var/ssl/server-key.pem',
  'certfile': '/var/ssl/server-crt.pem',
  'ca_certs': '/var/ssl/ca-crt.pem',
  'cert_reqs': ssl.CERT_REQUIRED
}

defining certificates as described in https://stackoverflow.com/a/34712536/6153117 but when running celery -A backend worker I got the error

[2018-03-04 16:27:16,771: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:645).
Trying again in 2.00 seconds...

[2018-03-04 16:27:18,794: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:645).
Trying again in 4.00 seconds...
DmitriyM
  • 125
  • 2
  • 9

2 Answers2

3

For celery ssl, ssl_listeners port is 5671 and you have mention 5672 in broker_url which is for TCP listeners. you need to change it.

I hope, It will help.

vermanil
  • 212
  • 1
  • 8
  • Now Connection refused error throws: `[2018-03-05 08:57:59,204: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5671//: [Errno 111] Connection refused. Trying again in 2.00 seconds...` – DmitriyM Mar 05 '18 at 09:24
2

In order to have your celery talking over SSL encrypted link you have to have your broker configured in such way that it will accept your client over SSL.

I use rabbitmq and by default it is not configured to handle SSL. There is a number of steps you have to take to enable SSL for it, you can look here for details:

You configure your rabbitmq to listen for ssl by changing your rabbitmq.config. You need following branches of config JSON configured:

  • ssl_listeners - this is where you specify what TCP port to listen on
  • ssl_options - this is where you specify your keys and certs and also TLS version
  • ssl - this is where you can specify what versions of TLS are enabled

It was already mentioned within another answer that rabbitmq will listen for SSL traffic at different port. You can see what ports are listened at running netstat -ntlp on your rabbitmq server.

The key point here is that your keys must be coming from your CA authority. Only valid keys will be allowed and also some options within rabbitmq will influence your process of issuing your keys and certs.

In the end we did not use rabbitmq's ssl capability and decided to route all traffic that needs protection over vpn link and let vpn to encrypt the traffic. For us it was a better move mainly because we had more services that could benefit from such VPN link so we did not have to maintain many CA authorities for many services.

Greg0ry
  • 931
  • 8
  • 25
  • can you describe a bit more about re-routing traffic over a VPN link? I am banging my head trying to figure out how to get celery to work with Rabbitmq over SSL. When I do `netstat -nltp`, I do not see Rabbitmq listening on port 5671 – lollerskates Aug 12 '19 at 06:04
  • Editing my original comment as I noticed there was a typo in it. In my setup I had a number of servers with private link available between them. I have configured `wireguard` mesh between those hosts using private links, and then I bound rabbitmq to interface brought up by `wireguard`. I added iptables rules to only pass relevant traffic through those interfaces. This setup worked for me and was simpler to maintain than to work with ssl certificates and directly with rabbitmq configuration.. – Greg0ry Jul 20 '21 at 19:03