How to use paho mqtt client in django?

Question

I am writing a django application which should act as MQTT publisher and as a subscriber.

Where should I start the paho client and run loop_forever() function.

Should it be in wsgi.py ?

`loop_forever()` will block forever, so it should probably be started in it's own thread — hardillb, Dec 07 '16 at 11:22

Max Polezhaev · Accepted Answer · 2018-02-04T03:33:43.470

23

Update:

If you need Django running in multiple threads then to publish messages from your Django app you can use helper functions from Publish module of Paho - https://eclipse.org/paho/clients/python/docs/#id17 You don't need to create an instance of mqtt client and start a loop in this case. And to subscribe to some topic consider running mqtt client as a standalone script and import there needed modules of your Django app (and don't forget to setup the Django environment in the script).

The answer below is good only if you run Django in a single thread, which is not usual in production.

Create mqtt.py in your application folder and put all related code there. For example:

import paho.mqtt.client as mqtt

def on_connect(client, userdata, rc):
    client.subscribe("$SYS/#")

def on_message(client, userdata, msg):
    # Do something
    pass

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect("iot.eclipse.org", 1883, 60)

Don't call loop_forever() here!

Then in your application __init__.py call loop_start():

from . import mqtt

mqtt.client.loop_start()

Using loop_start() instead of loop_forever() will give you not blocking background thread.

edited Feb 04 '18 at 03:33

answered Dec 07 '16 at 12:13

Max Polezhaev

281
2
7

Thanks Max, works fine !! I have another doubt, what happens if I scale the web process does that create multiple mqtt clients ? If so how do I stop that ? – Raja Sudhan Dec 07 '16 at 13:05
@RajaSudhan what do you mean by "scale the web process"? – Max Polezhaev Dec 07 '16 at 13:32
While running the django application in production I am deploying in multiple server with load balancer, hence I dont want to listen to messages in all the hosts. – Raja Sudhan Dec 15 '16 at 17:13
@RajaSudhan ok, now I understand your concern. You are right, if you run Django in multiple threads (as it usually happens in production) then you should not start a subscriber in every thread, because on_message in this case will fire in every thread for a single message, but you want it to fire just once. Consider running mqtt client as a standalone script and import there needed modules of your Django app (don't forget to setup Django environment in that script). – Max Polezhaev Dec 16 '16 at 17:09
1

@RajaSudhan my previous comment was about subscribing. And for publishing from Django you can use the helper functions from the Publish module of Paho, see updates on my answer. – Max Polezhaev Dec 16 '16 at 17:38
this work fine with ./manage.py runserver and not with: /usr/local/bin/uwsgi --socket :7011 --module myproject.wsgi Why? – user2239318 Mar 08 '17 at 13:28
@user2239318, can you provide more details? Do you see any exceptions? Can you share the code of your project? Also keep in mind, that the code in my answer is appropriate only if you run Django in a single thread, see updated part of the answer. – Max Polezhaev Mar 09 '17 at 17:13
When I do this, I actually run 3 times the script in production, does someone have any answers_ – Javier Campos Sep 18 '17 at 21:29
@JavierCampos what script do you run 3 times? Could you provide more details? – Max Polezhaev Sep 19 '17 at 22:50
@MaxPolezhaev I am usin gunicorn, in the settings I have 3 workes, so the script runs 3 times... any way to avoid this?(when I set to 1 worker, everything works fine) – Javier Campos Sep 27 '17 at 20:52
@JavierCampos you need a standalone script that is not imported into your Django app. You will have to execute that script with systemd or whatever you use (not from Django that is started 3 times by workers). – Max Polezhaev Oct 06 '17 at 09:22
@MaxPolezhaev and I would be able to write in the Django db with the django functions? – Javier Campos Oct 07 '17 at 21:47
@JavierCampos Yes, you will be able to use Django ORM and all your models in a standalone script. But don't forget to setup Django environment in the script (see https://stackoverflow.com/a/41266715/6408646). – Max Polezhaev Oct 13 '17 at 20:25
I followed this question but the django server stops and I cant acces the `localhost:8000`. please check my question: https://stackoverflow.com/questions/66417806/how-to-implement-paho-python-client-in-django-3-1 – Shahriar.M Mar 01 '21 at 07:27

Santiago Ivulich · Answer 2 · 2021-12-27T18:10:48.430

If you are using ASGI in your Django application you can use MQTTAsgi.

It's a complete protocol server for Django and MQTT.

Edit: Full disclosure I'm the author of MQTTAsgi.

Edit 2: To utilize the mqtt protocol server you can run your application, first you need to create a MQTT consumer:

from mqttasgi.consumers import MqttConsumer
class MyMqttConsumer(MqttConsumer):

    async def connect(self):
        await self.subscribe('my/testing/topic', 2)

    async def receive(self, mqtt_message):
        print('Received a message at topic:', mqtt_mesage['topic'])
        print('With payload', mqtt_message['payload'])
        print('And QOS:', mqtt_message['qos'])
        pass

    async def disconnect(self):
        await self.unsubscribe('my/testing/topic')

Then you should add this protocol to the protocol router:

application = ProtocolTypeRouter({
      'websocket': AllowedHostsOriginValidator(URLRouter([
          url('.*', WebsocketConsumer)
      ])),
      'mqtt': MyMqttConsumer,
      ....
    })

Then you can run the mqtt protocol server with*:

mqttasgi -H localhost -p 1883 my_application.asgi:application

*Assuming the broker is in localhost and port 1883.

Thanks for the comments!

Should I see some messages in the terminal running that `mqttasgi` command? I'm not seeing those "Received a message at topic:" messages. Wondering if I'm looking in the wrong place. I have tried other things that work. One maybe important note is that I'm using TLS encryption, so my command has certificates and stuff in addition to the host and port — Tsaari, Feb 10 '23 at 02:14
Nevermind. I changed my example to match the details on PyPI instead of the answer here. There were errors because the examples in this answer are outdated. — Tsaari, Feb 10 '23 at 02:47

How to use paho mqtt client in django?

2 Answers2

Linked

Related