2

I've a following up question to What is the stack size of a BackgroundWorker DoWork Thread? Is there way to change it?

Should I increase the stack size of my main program using the following post build event:

"$(DevEnvDir)..\..\VC\bin\editbin.exe" /STACK:8388608 "$(TargetPath)"

or should I capsule my recursive code block inside a new thread with a larger stack size?

Thread thread = new Thread(delegate()
{
    // do work with larger stack size
}, 8192 * 1024);
thread.Start();
thread.Join();

The code which has a lot of recursion is from Intel MKLs the LAPACKE_dtrtri function which I call over DLLImport. So I cannot change this code. I can just change the stack size to avoid a stack overflow error.

  • What is the draw back of assign a larger stack size to my main program?

  • What is the draw back of creating a new thread with a larger stack size for this calculation?

  • Which solution is the better one?

  • What would be a reasonable stack size? 8 MB, 16 MB, 32 MB or 64 MB (like MATLAB)? The program runs on computers which have at least 16 GB of RAM (up to 256 GB).

Note: for 99% of my application the default stack size of 4 MB (64-bit) would be enough but the other 1% is using external code which is heavily recursive.

Wollmich
  • 1,616
  • 1
  • 18
  • 46
  • None of these. As Hans Passant commented in that other question matrix inversion isn't rocket science, stack size is controlled by the OS and **NO operating system** is going to give you that much stack space. Big arrays aren't allocated on the stack and 16MB is *not* big. MATLAB doesn't freeze the OS when working with large arrays, which is what would happen if it used hundreds of MBs of stack space – Panagiotis Kanavos Jun 06 '19 at 11:21
  • In any case, SIMD commands were first written to deal with multimedia files, which are several times larger than the arrays you described. The programs that use them do this is by *not* using recursion over all the data. They work on data streams, and even disable the CPU cache since they know that the same data won't be needed again, so why perform *two* reads when only one is useful? – Panagiotis Kanavos Jun 06 '19 at 11:25
  • @PanagiotisKanavos If you've large matrices for example 70000x70000 (about 36.5 GB) it's not a trivial anymore. There are several limits I've to take care about: for example `` and https://stackoverflow.com/questions/47830510/2d-array-with-more-than-655352-elements-array-dimensions-exceeded-supported. So I came up with calling directly Intel MKL. My array is not on the stack but the computational routine from Intel MKL is recursive. – Wollmich Jun 06 '19 at 11:30
  • 1
    You're kinda paralyzed by this problem. Beyond all of the SO posts you already found, but forgot to attribute, if you want to handle giant matrices then be sure to demand 64-bit code execution so you have plenty of address space. Project > Properties > Build tab. Asking for a 1GB stack is then not unreasonable. – Hans Passant Jun 06 '19 at 11:34
  • @HansPassant 1 GB for the main program in that case? Do you've any idea why MATLAB choose 64 MB stack size? I never get an stack overflow error in MATLAB using very larges matrices so far. – Wollmich Jun 06 '19 at 11:50
  • 1
    It just doesn't matter, flip a coin. If it doesn't land on the side that's easiest to do then flip again. You'll have to assume that MATLAB is a very well tested program that doesn't rely on clumsy fortran code. – Hans Passant Jun 06 '19 at 12:11
  • @HansPassant MATLAB is using as well Intel MKL in the background, see `version('-lapack')` which returns `Intel(R) Math Kernel Library Version 2017.0.31 Product Build 20170606 for Intel(R) 64 architecture applications, CNR branch AVX2 Linear Algebra PACKage Version 3.7.0`. – Wollmich Jun 06 '19 at 12:17

1 Answers1

0
  • Which solution is the better one?
  • Which solution is the better one?

The first approach using editbin in a post-build event is failing when using strong name key to sign the assembly. After the assembly was changed using editbin the verification of the signed assembly will fail. sn.exe -v assembly.exe will return Failed to verify assembly -- Strong name validation failed ...

See as well:

Using the AfterCompile event and resigning the assembly is a workaround (which I'm using now). The project file should contain the following lines:

  <Target Name="AfterCompile">
    <Exec Command="
&quot;$(DevEnvDir)..\..\VC\bin\editbin.exe&quot; /STACK:16777216 &quot;$(ProjectDir)obj\$(ConfigurationName)\$(TargetFileName)&quot;
&quot;$(FrameworkSDKDir)bin\NETFX 4.5.1 Tools\sn.exe&quot; -Ra &quot;$(ProjectDir)obj\$(ConfigurationName)\$(TargetFileName)&quot; &quot;$(SolutionDir)\STRONGNAME.snk&quot;
" />
  </Target>
  <PropertyGroup>
    <PostBuildEvent>REM "See AfterCompile for stack size and resigning"</PostBuildEvent>
  </PropertyGroup>

I was getting aware of the after compile event when I was reading the following answer: https://stackoverflow.com/a/22617361/7556646

The second approach but for the hole program and not only for the recursive code blocks would look like that:

static class Program
{
    [STAThread]
    static void Main(string[] args)
    {
        Thread thread = new Thread(delegate()
        {
            Main2(args);
        }, 16 * 1024 * 1024);
        thread.SetApartmentState(ApartmentState.STA);
        thread.Start();
        thread.Join();
    }

    static void Main2(string[] args)
    {
        Application.EnableVisualStyles();
        Application.SetCompatibleTextRenderingDefault(false);
        Application.Run(app);
    }
}

The second approach has the drawback that the stack size of a BackgroundWorker DoWork event is still 1 MB (32-bit or any) or 4 MB (64-bit).

See as well:

  • What is the draw back of creating a new thread with a larger stack size for this calculation?
  • What would be a reasonable stack size?

See the comments from Hans Passant.

Wollmich
  • 1,616
  • 1
  • 18
  • 46