Integrate Flask with Faust

Question

I'm trying to get a faust agent to cast a message inside a flask view/endpoint, I can't find any example of it and i'm really struggling.

Has anyone tried this successfully? the docs say to use gevent or eventlet as a bridge to asyncio but can't understand how unfortunately

score 1 · Accepted Answer · answered May 03 '19 at 17:30

1

You can try to monkeypatch your flask application with gevent or eventlet. As far as I know eventlet is the only one which is currently working since the bridge aiogevent is no longer available in pip.

Another approach is to delegate your Flask app to only produce messages with pykafka or kafka-python. In this way you will keep both applications isolated, flask producing and faust consuming. Note that by default faust will try to create topics without internal flag with 1 replica and internal topics with 8 replicas.

answered May 03 '19 at 17:30

andre

192
2
12

thanks, in the end i went for the solution #2 with kafka-python and seems to be working fairly well for our use case. would still be nice to see the implementation of #1 as i literally have no clue – shipperizer May 08 '19 at 07:06
@shipperizer Can you please help me here with the solution with the first approach if not 2nd also works for me. Looking for the solution since ages and I am stuck right now. – Sahil Paudel Aug 24 '20 at 17:16
@sahil-paudel simply add a kafka producer to the flask blueprint at startup/bootstrap, then call it from.tour endpoint via app.kafka and use the send method. It s completely separated from Faust, we made it sharing the same marshmallow object so stuff sent to kafka is the same – shipperizer Aug 25 '20 at 19:40
Can you show me a sample code if you don't mind. Did you use faust at all? I am little uncomfortable using faust – Sahil Paudel Aug 26 '20 at 17:23

score 0 · Answer 2 · answered Jun 12 '19 at 23:11

The faust docs have a suggestion for integrating flask and faust. It involves using gevent, but I do not believe it requires monkeypatching.

From the docs (https://faust.readthedocs.io/en/latest/faq.html):

This approach works with any blocking Python library that can work with gevent.

Using gevent requires you to install the aiogevent module, and you can install this as a bundle with Faust:

$ pip install -U faust[gevent]
Then to actually use gevent as the event loop you have to either use the -L option to the faust program:

$ faust -L gevent -A myproj worker -l info
or add import mode.loop.gevent at the top of your entry point script:

#!/usr/bin/env python3
import mode.loop.gevent
REMEMBER: It’s very important that this is at the very top of the module, and that it executes before you import libraries.

Integrate Flask with Faust

2 Answers2