5

I have the following defined in a celery module named tasks.py with the requests library imported:

@celery.task
def geturl(url):
    res = requests.get(url)
    return res.content

Whenever I call the task (either from tasks.py or REPL) with:

res = geturl.delay('http://www.google.com')
print res.get()

Here are the log entries on the celery server:

[2012-12-19 18:49:58,400: INFO/MainProcess] Starting new HTTP connection (1): www.google.com [2012-12-19 18:49:58,594: INFO/MainProcess] Starting new HTTP connection (1): www.google.ca [2012-12-19 18:49:58,801: INFO/MainProcess] Task tasks.geturl[48182400-7d67-466b-ba5c-867747cb3974] succeeded in 0.402245998383s: None

But when I run this in a synchronous fashion by calling geturl('http://www.google.com'), I get a response. I have read through the docs and cannot seem to clue into why this is not working. The primary use of this is to poll API's could someone please explain why this does not work?

Thanks!

cdlm
  • 565
  • 9
  • 20

1 Answers1

1

res.content is just an str instance, there's no reason you wouldn't be able to return it. Your issue likely lies elsewhere, probably in your celery configuration.

Many configuration parameters play a role in how the results will be stored; have a look at them.
The first one you'll want to check may be CELERY_IGNORE_RESULT.

Your should also check your result broker configuration.

Thomas Orozco
  • 53,284
  • 11
  • 113
  • 116
  • Thank you, this seemed to work... The task seems to return the correct response (as indicated by the log entry below): [2012-12-19 22:40:06,580: INFO/MainProcess] Task tasks.geturl[52746098-f651-4627-96a5-ae50024a2b98] succeeded in 0.360215902328s: ' – cdlm Dec 20 '12 at 03:42
  • I believe the above ^ was happening because I had celery.worker_main() executing in the same module that I had the task defined in so the task never had time to run since the event loop was blocking the thread. Once I moved this out of the worker code, everything works as expected. – cdlm Dec 20 '12 at 03:54