1

I've a python based SSRS report generation utility that I'm using to generate multiple reports (often 100+). The way it's setup is -

  1. Multiple threads are invoked using threading.Thread and each of them is given a dictionary.
  2. Each thread parses the dictionary and calls rs.exe passing in relevant arguments via python's subprocess.call

Reports get generated with the following caveats -

  1. If there are around 20-30 reports everything works fine without much issues.
  2. If the number of reports go beyond 40-50+ (for reasons unknown to me so far), some of the reports don't get rendered and come back with error as obtained by subprocess.call non-zero status (Error message from subprocess.call does not point to any real error). But there is no error in those rs.exe commands, as they get rendered when i run them from windows command prompt.
  3. Additionally when i try to re-run all those failed reports they get rendered. There's no change in the commands or data while they're being re-run.

To work around this, I employed a retry logic for 2 iterations which seems to fix the issue at times. However when the reports go beyond 100/150+ even the retry doesn't work. Now i could extend the retry logic to keep retrying until all the reports are rendered and whatever failures happen are genuine ones (like RDL not found, corrupted and so on). But before i do any such thing, want to know if there's any limitation on how many rs.exe can be launched simultaneously or if there's any limitation on python's subproces.call when invoked in a multi-threaded context.

Can someone please share their expertise if they've faced this kind of issue and resolved it?

Thanks.

Kevin
  • 74,910
  • 12
  • 133
  • 166
Murali
  • 111
  • 1
  • 3
  • 9
  • If there does happen to be a limit on the number of simultaneous subprocess calls, perhaps you could use a [semaphore](https://docs.python.org/2/library/threading.html#semaphore-objects) to make sure no more than, say, 20 threads can execute `call` at the same time. – Kevin Dec 08 '15 at 19:44
  • Thanks for the suggestion. I changed the implementation to use ThreadPoolExecutor with max of 20 threads and it worked, atleast in the first run. No errors and didn't have to do any retry. Will do some more test cases to make sure behavior is consistent. – Murali Dec 08 '15 at 21:02

1 Answers1

0

I suspect the limit you are hitting is not rs.exe itself but the target Report Server. This will use as much physical memory as is available but when that is exhausted, further requests will start to fail. This is described in the SSRS doco:

https://msdn.microsoft.com/en-us/library/ms159206.aspx

To avoid this issue and leave some server resources for other users, I would reduce your thread limit as low as you can stand - ideally to 1.

Mike Honey
  • 14,523
  • 1
  • 24
  • 40