21

I am investigating for how many time it takes for a particular operation to complete. The operation is like the following:

Parallel.ForEach(items, item => SaveScheme(item));

The SaveScheme method works with a database: executes some queries and works with the information. The amount of elements in items collection can be big enough.

When I run this operation, it takes about 20-40 seconds to complete. But when I run it with a profiling turned on, it takes only 3 seconds!

I didn't find any information about this problem. My only guess is that with profiling Parallel.ForEach creates more threads than without it, but I don't know for sure, and even if it's true, I don't know what to do with it.

So, why is that happens and how can I archieve this performance when I run the application without profiling?

UPD. Parallel has nothing to do with this: I tested with simple foreach instead and the operation still completes in 3 seconds!

STiLeTT
  • 1,023
  • 10
  • 23
  • hmm Strange, actually it will slow down... – Anuraj Aug 16 '12 at 09:47
  • HAve you tried the Concurrency Visualizer ? – h1ghfive Aug 16 '12 at 09:50
  • I have tried the Concurrency Visualizer, but what I saw there tells me nothing. I know how many threads there are and so on, but without profiler I don't know what happens when the operation takes 40 seconds. So, I can't compare it. – STiLeTT Aug 16 '12 at 10:39
  • Are you sure that the database responses in exactly the same way (speed, resultset) in both situations and is not using cached results or prepared statements? – Emond Aug 17 '12 at 09:55

5 Answers5

38

I found the answer:

The reason is because when you run your application within Visual Studio, the debugger is attached to it. When you run it using the profiler, the debugger is not attached.

If you try running the .exe by itself, or running the program through the IDE with "Debug > Start Without Debugging" (or just press Ctrl+F5) the application should run as fast as it does with the profiler.

https://stackoverflow.com/a/6629040/1563172

I didn't find it earlier because I thought that the reason is concurrency.

Community
  • 1
  • 1
STiLeTT
  • 1,023
  • 10
  • 23
  • Note that if you like running your scripts in the interactive console (e.g. to get the benefits of IPython), you can detach the debugger through Python Tools -> Interactive Window -> untick "Enable attaching to interactive window", and get the same speed up. – Leo Aug 16 '15 at 15:04
1

Since you are using threading in your program, the Windows timer resolution can also be a reason.

Default windows timer resolution is 15.6ms

When you run your application with the profiler, this is reduced to 1ms causing your application to run faster. Checkout this answer

Kusal Dissanayake
  • 714
  • 14
  • 32
0

I suspect what you are actually profiling is only the 'not parrarel' part of your application.

I would suggest to take a look here : http://msdn.microsoft.com/en-us/library/gg663532.aspx

h1ghfive
  • 196
  • 7
  • No, in my application it is clearly seen when the operation has been completed, and yes, it completes after 3 seconds. – STiLeTT Aug 16 '12 at 09:55
0

Could it be that in profiling you have slowed down (and decreased the concurrency) of the actual database work generation that you have effectively throttled the bottleneck itself - yes the database (or ....).

William Louth
  • 207
  • 1
  • 4
0

This result suggests that your application is perhaps trying to be too concurrent: your threads are in fact getting in eachothers' way, or the overhead of creating your threads is greater than any performance gain. This is evidenced by the fact that your non-concurrent version runs faster!

The use of the profiler will affect the performance of your application; perhaps it is slowing down your code enough that you do see some benefit from multiple threads.

Without more detail of the code behind your method, this seems the most likely answer.

Dan Puzey
  • 33,626
  • 4
  • 73
  • 96