6

I have a python web application written using FastAPI (running via uvicorn). In my app I'm using the standard logging module which uses TimedRotatingFileHandler. Since I'm logging into a file I'm concerned about performance. Lets say I have 3 log messages in a function which is called with every call to a /test endpoint. Now imagine 1000 clients requesting /test at the same time - that is 3000 writes to the log file.

I'm concerned this will hinder the app's performance since IO tasks are time consuming. Should I use some form of asynchronous logging? Maybe open a new thread which will write into the file? I tried googling for answers but I found no standardized approach to it. So is it even needed? If yes, how should I approach this? Thanks!

muliku
  • 416
  • 3
  • 17
  • 1
    My2cts, avoid premature optimization. Is your API too slow for you needs ? Is it too slow because of logging. If not don't over-optimize it. – Emmanuel-Lin Apr 13 '22 at 12:37
  • 1
    Also 1000 calls to a single instance of your application is going to be difficult to scale anyways. Ideally you would scale horizontally with multiple instances of your web application on different VMs/containers to achieve higher throughputs. – flakes Apr 13 '22 at 12:44
  • 3
    Please have a look at [this answer](https://stackoverflow.com/a/70899261/17865804), which uses a [`background task`](https://fastapi.tiangolo.com/tutorial/background-tasks/) for logging the data (hence, logging takes place after a response is returned). If you need to start a new Thread inside the `background task` function, have a look at [this answer](https://stackoverflow.com/a/71738644/17865804). Note: Asynchronous doesn't mean multithreading. Please take a look [here](https://stackoverflow.com/a/71517830/17865804) to understand the concept of `async`/`await`. – Chris Apr 13 '22 at 14:56

0 Answers0