How to integrate Faust with Django?

Question

I am trying to integrate Faust with Django to publish the messages to Kafka. Here is the example in Faust repo: https://github.com/robinhood/faust/tree/master/examples/django

I modified it a bit, and created views to push data to Kafka via Faust.

from django.shortcuts import render

from asgiref.sync import async_to_sync

from accounts.agents import AccountRecord, add_account


async def send_data() -> None:
    print("sending..data")
    print(await add_account.ask(AccountRecord(name="tesst", score=10.9, active=False)))

def index(request):
    async_to_sync(send_data)()
    return render(request, "accounts/index.html")

But, I get this error now:

RuntimeError at / Task <Task pending name='Task-1' coro=<AsyncToSync.main_wrap() running at /Users/mysuer/.pyenv/versions/3.8.3/envs/faustdjango/lib/python3.8/site-packages/asgiref/sync.py:204> cb=[_run_until_complete_cb() at /Users/mysuer/.pyenv/versions/3.8.3/lib/python3.8/asyncio/base_events.py:184]> got Future attached to a different loop

I am running this Django app using development server. What am I doing wrong? Anyone? :)

https://stackoverflow.com/questions/53724665/using-queues-results-in-asyncio-exception-got-future-future-pending-attached — Siva Sankar, Jan 26 '21 at 11:38
would you provide full code. And also, above link has same bug and answer. Did you check it? — Siva Sankar, Jan 29 '21 at 16:39
@Maverick, how are you executing this in Django as the main app? As per the example of [faustapp](https://github.com/robinhood/faust/blob/master/examples/django/faustapp/app.py) they ensure to use the event loop by making sure the app which starts off uses the`faust.App()` — Nagaraj Tantri, Feb 23 '21 at 08:47
I believe that it happens when you try to run a function that execute itself in different thread (or process). As i can see in faustapp source code it is defined as async function so you can not call it as sync one. — SaGaR, Feb 24 '21 at 04:17

score 2 · Answer 1 · answered Feb 25 '21 at 02:36

See the FAQ section in faust documentation

Can I use Faust with Django/Flask/etc.?

Yes! Use eventlet as a bridge to integrate with asyncio.

Using eventlet

This approach works with any blocking Python library that can work with eventlet.

Using eventlet requires you to install the aioeventlet module, and you can install this as a bundle along with Faust:

$ pip install -U faust[eventlet]

Then to actually use eventlet as the event loop you have to either use the -L <faust --loop> argument to the faust program:

$ faust -L eventlet -A myproj worker -l info

or add import mode.loop.eventlet at the top of your entry point script:

#!/usr/bin/env python3
import mode.loop.eventlet  # noqa

Warning It's very important this is at the very top of the module, and that it executes before you import libraries.

This doesn't help at all, as it still doesn't explain how to send calls to agents from a view. None of the documentation does - not even their Django example. — Roger Collins, Apr 29 '21 at 21:51
I am curious, with more async support to django views and orm's should we revisit faust integration examples with django? — auvipy, May 29 '22 at 18:05

How to integrate Faust with Django?

1 Answers1