I have a 500 MB networkx graph pickle object. Celery workers need this graph on each HTTP request to do some computation in background. I want to upload this graph only once in the memory on django startup. How can I do it most efficiently?
Asked
Active
Viewed 141 times
2
-
Do you need a graph on celery side or on django? For celery worker solution check this question - http://stackoverflow.com/questions/2129820/how-can-i-set-up-celery-to-call-a-custom-initialization-function-before-running – kmmbvnr Jun 16 '15 at 07:50
-
Thanks for sharing this link. If I choose celery worker solution approach and upload the graph on worker_init() , then how can I access this graph within the @shared task function? – Jun 16 '15 at 11:29
-
As in the the sample - you can use global variable. – kmmbvnr Jun 16 '15 at 15:04