Multiprocessing, Threading, Pyside, Django Apps aren't loaded yet

Question

I'm trying to get pyside, multiprocessing/threading and django work together. I have a function "update_source_product" that is called upon a push button of django object actions.

class BrowserActions(object):
    def update_source_product(self, request, obj):
        mongo_link = Link.objects.get(_id=ObjectId(obj.source))
        p = multiprocessing.Process(target=GET.main, args=(mongo_link.link, ))
        p.start()
        p.join()
        mongo_link.save()

Following are Pyside imports

from PySide.QtGui import *
from PySide.QtCore import *
from PySide.QtWebKit import *

In the function a process is supposed to be started that shows a QApplication on the server which is defined in the following function.

class GET(object):
    def __init__(self):
        pass

    @classmethod
    def main(self, url):
        app = QApplication(sys.argv)
        web = QWebView()
        web.load(QUrl(url))
        web.show()
        sys.exit(app.exec_())

Now when I run this code as a thread with

        t = threading.Thread(target=GET.main, args=(mongo_link.link, ))
        t.daemon = True
        t.start()
        t.join()

a window pops up and shows the page

but when I close the window, Windows throws an error "python.exe has stopped working"

and django dev server crashes. The stack trace is as following.

WARNING: QApplication was not created in the main() thread.
QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once.
QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once.
content-type missing in HTTP POST, defaulting to application/x-www-form-urlencoded. Use QNetworkRequest::setHeader() to fix this problem.
content-type missing in HTTP POST, defaulting to application/x-www-form-urlencoded. Use QNetworkRequest::setHeader() to fix this problem.
content-type missing in HTTP POST, defaulting to application/x-www-form-urlencoded. Use QNetworkRequest::setHeader() to fix this problem.
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
QObject::killTimers: timers cannot be stopped from another thread

And When I run it as a separate process with

        p = multiprocessing.Process(target=GET.main, args=(mongo_link.link, ))
        p.start()
        p.join()

The Window never shows up but the django dev server keeps running and prints the following stack trace

File "<string>", line 1, in <module>
File "C:\Python27\Lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
File "C:\Python27\Lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
File "C:\Python27\Lib\pickle.py", line 864, in load
    dispatch[key](self)
File "C:\Python27\Lib\pickle.py", line 1096, in load_global
    klass = self.find_class(module, name)
File "C:\Python27\Lib\pickle.py", line 1130, in find_class
    __import__(module)
File "C:\Users\Snst\PycharmProjects\isp_data_core\data\tasks.py", line 1, in <module>
    from data.models import Product, Pattern, Campus_Product, Campus, \
File "C:\Users\Snst\PycharmProjects\isp_data_core\data\models.py", line 2, in
<module>
    from django.contrib.auth.models import User
File "C:\Users\Snst\PycharmProjects\isp_data_core\isp_data_core_venv\lib\site-packages\django\contrib\auth\models.py", line 4, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "C:\Users\Snst\PycharmProjects\isp_data_core\isp_data_core_venv\lib\site-packages\django\contrib\auth\base_user.py", line 49, in <module>
    class AbstractBaseUser(models.Model):
File "C:\Users\Snst\PycharmProjects\isp_data_core\isp_data_core_venv\lib\site-packages\django\db\models\base.py", line 94, in __new__
    app_config = apps.get_containing_app_config(module)
File "C:\Users\Snst\PycharmProjects\isp_data_core\isp_data_core_venv\lib\site-packages\django\apps\registry.py", line 239, in get_containing_app_config
    self.check_apps_ready()
File "C:\Users\Snst\PycharmProjects\isp_data_core\isp_data_core_venv\lib\site-packages\django\apps\registry.py", line 124, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet

Can someone please suggest a solution or point out what am I doing wrong in QApplication life-cycle as a process/thread.

I have no idea **why** you would want to do this, but I suspect that you need to replace `sys.exit(app.exec_())` with `app.exec_()` because you don't want the program to end when the Qt main loop finishes. — three_pineapples, Jul 18 '16 at 09:28
I'm not sure I agree this is a duplicate. The solution in the duplicate post is unlikely to apply here as the error is only visible when using the multiprocessing library. This is likely because windows doesn't fork the same way linux does and the pretend "fork" under windows is partially running some django things. Implementing the fix for the forked copy is likely to break the original copy unless you carefully apply the fix inside an `if __name__ != "__main__"` block. Either way, this question is no where near as simple as the duplicate due to the mixing of django with pyside. — three_pineapples, Jul 18 '16 at 10:48
It is that simple. The OP starts a separate process with a different entry point (not `manage.py` or `get_wsgi_application()`), and uses Django models in that process, so that entry point must manually call `django.setup()`. The exact same error would also happen if the QApplication was started as a stand-alone process rather than a subprocess, it is not related to forking at all. It's not "only visible when using the multiprocessing library", it's just not visible when using the multi-threading library, because `django.setup()` populates a process-wide application registry. — knbk, Jul 18 '16 at 11:10
yes it is not a duplicate and the "django.setup()" solution didn't work for me. the The goal is to run a QApplication main loop as a separate process/thread alongside django similar to selenium, so that django server doesnt have to wait for QApplication's main loop to finish. And as any browser in selenium takes a lot of resources so I'm looking for a lightweight substitute. Piping between the two processes is a later issue which I will come to next. — , Jul 18 '16 at 11:26
If you can elaborate in your question what you tried and why it didn't work, I will reopen it. Right now it doesn't give enough information to say anything other than "use `django.setup()`". — knbk, Jul 18 '16 at 11:33
This is a strange website. BTW if can't understand the question at least don't mark it as duplicate !!! And I'm talking about the solution mentioned in the following question http://stackoverflow.com/questions/34114427/django-upgrading-to-1-9-error-appregistrynotready-apps-arent-loaded-yet — , Jul 18 '16 at 11:55
So where did you put that call to `django.setup()`? It may help to show the module that defines `GET.main` (including imports) and to tell us what type of object `mongo_link.link` is. That might shed some light on what kind if issues you're having. — knbk, Jul 18 '16 at 12:07
I tried in two places one in wsgi.py after `application = get_wsgi_application()` and second before starting the thread or process `t = threading.Thread(target=GET.main, args=(mongo_link.link, ))` or `p = multiprocessing.Process(target=GET.main, args=(mongo_link.link, ))` — , Jul 18 '16 at 12:12
Well, you need to call it in the new process you're spawning with `multiprocessing.Process`, before you import any models. Calling it in the parent process won't have any effect. — knbk, Jul 18 '16 at 12:14
I called it in the "main" function which is "the process" but has no effect. In-fact the qapplication works nicely when I use threading but when I close the window and throws "python.exe has stopped working" error. If I could terminate the QApplication and the thread properly and not affect django dev server the goal is achieved. — , Jul 18 '16 at 12:22
This question is closed as a duplicate, and I know it's years old. But, it's definitely not a duplicate. For anybody else experiencing this problem, multiprocessing pickles your data in order to pass arguments to your entry point. If one of those arguments is, say, a Django model, or it is defined in a module that directly or indirectly loads Django models or translations, the above described problem will happen because pickling needs to import that type to unpickle it. Try to keep arguments to the function to only concrete types, like `str` or `int`. — Brett Gmoser, May 31 '23 at 22:49

Multiprocessing, Threading, Pyside, Django Apps aren't loaded yet

0 Answers0