2

I am trying to follow the instructions in http://flask.pocoo.org/docs/0.12/patterns/celery/ so I can perform flask/socketIO operations in celery tasks. My directory structure is a bit different however and I'm not having any luck with imports.

My directory structure is as follows:

├── app
│   ├── __init__.py
│   ├── __pycache__
│   ├── auth.py
│   ├── ctasks.py
│   ├── helper.py
│   ├── saml.py
│   ├── socks.py
│   ├── templates
│   ├── threads.py
│   └── views.py
├── app.py
├── config.py
├── requirements.txt
└── saml
    ├── dev
    └── prod

I call the app from app.py

from app import socketio, app

if __name__ == '__main__':
    socketio.run(app, debug=True, port=443, ssl_context='adhoc')

__init__.py

from flask import Flask, request
from flask_socketio import SocketIO
from .ctasks import subtaskcaller, make_celery
from .helper import wait_to_finish

async_mode = None

app = Flask(__name__)
app.config.from_object('config')
socketio = SocketIO(app, async_mode=async_mode)
cel = make_celery(app)

from .auth import SamlManager
saml_manager = SamlManager()
saml_manager.init_app(app) 
from app import views, socks, saml, helper, ctasks

ctasks.py

from celery import Celery
from config import *
from .helper import wait_to_finish, emitter
import time
from app import cel


def make_celery(app):
    c = Celery(app.import_name, backend=CELERY_RESULT_BACKEND, broker=CELERY_BROKER_URL)
    c.conf.update(app.config)
    taskbase = c.Task

    class ContextTask(taskbase):
        abstract = True

        def __call__(self, *args, **kwargs):
            with app.app_context():
                return taskbase.__call__(self, *args, **kwargs)

    c.Task = ContextTask
    return c

@cel.task(name='tasks.tester', serializer='pickle')
def tester():
    emitter('emit from subsubtask')
    for i in range(1, 50):
        time.sleep(1)
        print('test {0}'.format(i))
    x = True
    return x

@cel.task(name='task.subtaskcaller', serializer='pickle')
def subtaskcaller():
    emitter('emit from subtask')
    finished = tester.delay()
    wait_to_finish(finished)
    return finished

I am getting an error when trying to import cel from app in ctasks.py:

ImportError: cannot import name 'cel'

user1601716
  • 1,893
  • 4
  • 24
  • 53
  • Possible duplicate of [What is \_\_init\_\_.py for?](https://stackoverflow.com/questions/448271/what-is-init-py-for) – Morse Apr 18 '18 at 21:44
  • I believe I am using unit correctly and have used imports similar to this in the past. I am guessing I have something out of order, a circular import or import dependency problem. I had a typo in my original code and have updated. – user1601716 Apr 19 '18 at 00:11

1 Answers1

2

In your __init__.py you only have cel. You don't have an object called celery in your __init__ file, so it can't be imported to some other file. You can try from app import cel

EDIT:

in __init__ you from .ctasks import subtaskcaller, make_celery

but in ctasks you import cel from app (which doesn't exist yet at that point, only Flask, request, and SocketIO exist at that point in time).

So you need to put your @cel decorated functions in yet another script, which you can import all the way at the bottom of __init__

Joost
  • 3,609
  • 2
  • 12
  • 29