Is Celery needed for asynchronous file writing functionality in Django?

Question

I'm planning to add some functionality to my Django app that will write a row of data to a csv file when certain views are called:

    session_dict = collectUserDataFromRequest(request) #....collect parameters from  request

     if os.path.isfile(filename):

        with open(filename, 'a') as csv_file:
            a = csv.DictWriter(csv_file,session_dict.keys())
            a.writerow(session_dict)
            csv_file.close()

    else:

        with open(filename, 'w') as csv_file:
            w = csv.DictWriter(csv_file,session_dict.keys())
            w.writeheader()
            w.writerow(session_dict)
            csv_file.close() 

     # then proceed to render the html

A problem I am facing is dealing with potential race conditions when multiple users are accessing my app at the same time. While the data can be added to the csv file in any order, I don't want some user's to wait longer for a view to load while waiting for a lock to be released on the csv file. As such, I think an Asynchronous queue of string data to be written to the file that can run in the background sounds like a good workaround.

I have been told that Celery is a good framework for adding asynchronous processes to my Django app. But looking through several tutorials, I am seeing that additional frameworks such as Redis are usually required. I worry that this may be overkill for what I aim to accomplish. Will I get away with using a standard multiprocessing approach, such as explained here or are there other drawbacks to using multi-threading in a Django production environment.

Answer 1

First point: you should not have concurrent requests trying to write to the same file. Depending on your use case, you should either store your data in a proper relational database which knows how to handle concurrent access or have per-user files.

Second point: no, you don't necessarily needs celery for async operations, there are quite a few other solutions.