0

Our webserver generates a file on the fly for download. This takes about two minutes of heavy processing because it's taking 1000 word templates, mail merging them, converting them to pdf, then making a zip out of those 1000 files. The processing is freezing the web server from being able to do anything else in the mean time which is a problem since it's hosting 23 subdomains for clients and they noticed it freezing.

How can i force the UI thread to get some work done? I've been looking at Thread.Sleep and Thread.Yield but perhaps i'm not implementing them correctly. I'm very new to the concept of threading.

John Lord
  • 1,941
  • 12
  • 27
  • Use Async method to process the heavy workload – Anup Sharma Aug 14 '19 at 21:21
  • Thread.Sleep will stop the thread to sleep. Not good. – LarsTech Aug 14 '19 at 21:22
  • I tried processing the 1000 ebols asyncronously. It cut down on the processing time but maxed the cpu usage to 100% for a minute and a half. – John Lord Aug 14 '19 at 21:24
  • Was parallel.for a bad idea? – John Lord Aug 14 '19 at 21:27
  • You can't have it run quickly while also not consuming CPU. I think your question is too vague to really be answerable, but I can answer in broad terms. It's unclear why processing on the server would freeze the web client. –  Aug 14 '19 at 21:28
  • the "client" isn't freezing. The server is unresponsive to page loads during the processing. – John Lord Aug 14 '19 at 21:30
  • 1
    I suggest this sentence be rephrased so it is clearer, then: `The processing is freezing the web browser from being able to do anything else` –  Aug 14 '19 at 21:30
  • 2
    Then don't have your web server process files. It's not meant to do that. It's meant to handle HTTP requests. Have that work done in a scalable background processor application (possibly hosted as Windows service, or an Azure Function etc). Then you can spin up as many of those as you need to perform the work, while keeping your web server from becoming overloaded. – mason Aug 14 '19 at 21:36
  • thanks amy. That was a typo – John Lord Aug 14 '19 at 21:37
  • 1
    Mason I will look into your suggestion. I have constraints on how far into the weeds i'm allowed to take this stuff and we don't have an azure anything, but possibly we could offload the processing to our console application that uses the generation. – John Lord Aug 14 '19 at 21:39

1 Answers1

1

When starting the processing on the web server, generate a "job ID" and store it somewhere (such as a database). Add an endpoint so the client can query the status of the job. When the processing is complete, the user can use the job ID to get the resulting file(s). It works like this:

  • User wants to process files. They call the start endpoint, which returns a JobId.
  • The server begins processing that job in a non-request thread, or the job is picked up and processed by another server dedicated to that task. When the thread completes, it updates the job's status accordingly.

Later...

  • User wants to know the status of their process. They call the status endpoint using their JobId periodically. The server replies with some status information to show the user.

Later...

  • Once the job's status has changed to 'complete', the user can call the result endpoint with their JobId to get the final file(s).

The heavy processing should be done in a non-UI, non-request thread so other users are unaffected.

Using this approach, you can even do the processing on another server entirely. All the web server is doing is allowing the user to create and query processing jobs.

  • This is all true and I know this but i'm having trouble (evidently) getting it on a different thread. Does Parallel.For freeze the UI thread? – John Lord Aug 14 '19 at 21:35
  • 1
    The thread calling `Parallel.For` waits for `Parallel.For` to complete. –  Aug 14 '19 at 21:36
  • ok well i tried this: I put the function in a async function but i'm having the same exact problem i had before: The await never finishes. It's like it never knows the threads are finished. – John Lord Aug 14 '19 at 22:05
  • Ask a new question, John, and post some code. We cannot tell you what you're doing wrong when we cannot see your code. –  Aug 15 '19 at 01:01
  • I'm currently taking your advice and trying to offload the entire function into a different app. I will let you know how well it works. – John Lord Aug 15 '19 at 14:12
  • @JohnLord Please do, I'm interested in how things turn out. –  Aug 15 '19 at 14:25
  • I'm still debugging results, but initial testing is promising. Even with high cpu usage, the website is still responsive. – John Lord Aug 16 '19 at 22:03