1

I'm using dramatiq with flask. In dramatiq i have a function which send files to google drive. but there is an issue.

if i call dramatiq function only one time and wait for result by myself until all files is delivered to google drive it is ok.

But the problem is if i several times call my dramatiq function not waiting when dramatiq complete - it cause the situation when dramatiq do not wait for result for previous call and send function again. How to prevert this behavior? How i can instruct to dramatiq do not call function if another is not complete? Need to await for complete dramatiq first call before another? But do it asynchronous

my functions is:

@dramatiq.actor(store_results=True)
def my_actor(credentials, loan_number, id_folder):

    print('start')
    g = GoogleApi(credentials)
    directory = os.path.join(UPLOAD_FOLDER_MAIN + "/" + str(loan_number))
    for i in list_of_files:
         g.upload_file(i, id_folder)
 
    return True

I try to use get result with '''block true''' but it cause block I/O. how to do it asynchronous?

Thank you

dmitriy_one
  • 477
  • 1
  • 5
  • 16

1 Answers1

0

you can use pipelines as shown in the docs
maybe use message_with_options(pipe_ignore=True) if you dont need the result of the previous job
https://dramatiq.io/cookbook.html#pipelines
if the tasks dont need to be in order but just run only 1 at a time maybe rate limit would help
https://dramatiq.io/cookbook.html#rate-limiting

Tal Leibman
  • 306
  • 4
  • 4