Excel Add-in: Assignment in for loop causes segmentation fault but line-by-line assignments work. Why?

Question

I just tested a toy Excel add-in project, cross building the XLL with mingw32 tool chains.

Here is my code:

//testXLL.c
#include "windows.h"
#include "xlcall.h"

#define MEMORYSIZE 65535000
char vMemBlock[MEMORYSIZE];
int vOffsetMemBlock =0;

LPSTR GetTempMemory(int cBytes){
    LPSTR lpMemory;
    if(vOffsetMemBlock + cBytes > MEMORYSIZE)
        return 0;
    else{
        lpMemory = (LPSTR) &vMemBlock + vOffsetMemBlock;
        vOffsetMemBlock += cBytes;
        if(vOffsetMemBlock & 1) vOffsetMemBlock++;
        return lpMemory;
    }
}

LPXLOPER TempStr(LPSTR lpstr){
    LPXLOPER lpx;
    int chars;
    lpx = (LPXLOPER)GetTempMemory(sizeof(XLOPER));
    if(!lpx) return 0;
    chars = lstrlen(lpstr); 
    if(chars>255) chars=255;
    lpx->val.str=(char*)GetTempMemory((sizeof(char)*chars+1));
    if(!lpx->val.str) return 0;
    strncpy(lpx->val.str, lpstr,chars);
    lpx->val.str[0]=(BYTE) chars;
    //lpx->val.str[chars]='\0';
    lpx->xltype = xltypeStr;
    return lpx;
}
   
#ifdef __cplusplus
extern "C" {
#endif

    __declspec(dllexport) double __stdcall myadd2(double a1,double a2){
        return a1+a2;
    }

    static char functionTable[11][255] =
    {" myadd2",                    // procedure
        " BBB",                        // type_text
        " add",                     // function_text
        " add1,add2",                     // argument_text
        " 1",                          // macro_type
        " category",              // category
        " ",                           // shortcut_text
        " some help topic",                           // help_topic
        " Adds toy",    // function_help
        " 1st.",   // argument_help1
        " 2nd"   // argument_help2
    };

    __declspec(dllexport) int __stdcall xlAutoOpen(){
        LPXLOPER pxDLL;
        Excel4(xlGetName,pxDLL,0);

        XLOPER xlRegArgs[11];
        for(int i = 0; i < 11; i++){
            xlRegArgs[i] = *TempStr(functionTable[i]);
        }


        Excel4(xlfRegister, 0, 12,
                pxDLL,
                &xlRegArgs[0], &xlRegArgs[1], &xlRegArgs[2], 
                &xlRegArgs[3], &xlRegArgs[4], &xlRegArgs[5],
                &xlRegArgs[6], &xlRegArgs[7], &xlRegArgs[8],
                &xlRegArgs[9], &xlRegArgs[10]);

        return 1;
    }

    __declspec(dllexport) LPXLOPER __stdcall xlAddInManagerInfo(LPXLOPER xlAction) {
        static XLOPER xlReturn, xlLongName, xlTemp;

        xlTemp.xltype = xltypeInt;
        xlTemp.val.w = xltypeInt;
        Excel4(xlCoerce, &xlReturn, 2, xlAction, &xlTemp);

        if(1 == xlReturn.val.w) {
            xlLongName = *TempStr(" xll-name"); 
        } else {
            xlLongName.xltype = xltypeErr;
            xlLongName.val.err = xlerrValue;
        }

        return &xlLongName;
    }

#ifdef __cplusplus
}
#endif

I built this testXLL.c file in Ubuntu:

>i686-w64-mingw32-gcc -shared -Wl,--kill-at testXLL.c -o win.xll -L. -lxlcall32

This generates the "win.xll" successfully but, when loading this win.xll, Excel crashes.

In Windows 10, I tried to use gdb to debug it, but I can't catch break point in the xll file – it got disabled automatically when loading. But I can see in the gdb output, it is a segmentation fault when Excel crashes.

 XLOPER xlRegArgs[11];
for(int i = 0; i < 11; i++){
    xlRegArgs[i] = *TempStr(functionTable[i]);
}

What's weird is that, if I substitute the above for loop with the following line-by-line assignments in the xlAutoOpen function, the compiled XLL file works fine in Excel:

XLOPER xlRegArgs[11];
xlRegArgs[0] = *TempStr(functionTable[0]);
xlRegArgs[1] = *TempStr(functionTable[1]);
xlRegArgs[2] = *TempStr(functionTable[2]);
xlRegArgs[3] = *TempStr(functionTable[3]);
xlRegArgs[4] = *TempStr(functionTable[4]);
xlRegArgs[5] = *TempStr(functionTable[5]);
xlRegArgs[6] = *TempStr(functionTable[6]);
xlRegArgs[7] = *TempStr(functionTable[7]);
xlRegArgs[8] = *TempStr(functionTable[8]);
xlRegArgs[9] = *TempStr(functionTable[9]);
xlRegArgs[10] = *TempStr(functionTable[10]);

Please enlighten me. What's the difference between these two assignment approaches?

Minor: `if(chars>255) chars=255;` should be 254 to include terminating null. After `strncpy` you must terminate the string at index 254. — Paul Ogilvie, Apr 01 '21 at 10:19
@PaulOgilvie it should be 12, the first argument of xlfRegister is the full path of the xll,pxDLL. the other 11 arguments are exported function related. Since the line by line assignment version works, so other part of the codes should be ok. — user3148104, Apr 01 '21 at 10:37
I have had similar issues when building DLLs for x86. Try making your loop index (`i`) a `static` variable outside the function; if this works, then it looks like some sort of stack corruption issue. I'll dig a bit deeper to see if/how I ever resolved the problem. — Adrian Mole, Apr 01 '21 at 10:46
@AdrianMole That's point, Thank you! I just declare i as global variable, it never crashes! Is that a BUG of EXCEL? — user3148104, Apr 01 '21 at 10:59
Don't think it's an Excel issue. More likely the DLL build is not *quite* correct. Check all your project settings - alignment issues, calling conventions, etc. I have the same thing using extension DLLs called from my main program ... but only in x86 (not x64) builds. — Adrian Mole, Apr 01 '21 at 11:02
@AdrianMole This toy project just has only one source file, just as listed "testXLL.c",and explicitly define the __stdcall call convention. Is that the cross platform compiler's issue? — user3148104, Apr 01 '21 at 11:10
In MSVC I would add `__debugbreak();` before loop, specify Excel as host for debugging DLL and start debugging. When debugger hits breakpoint, set data trap on `i` and carefully step over (F10) few times, observing `i` value. This [https://stackoverflow.com/a/49079078/8666197](https://stackoverflow.com/a/49079078/8666197) could help you to add hard breakpoint in gcc. Alternatively use poor man debugging with `MessageBox` or `OutputDebugString` if gdb supports it. — Daniel Sęk, Apr 07 '21 at 06:44
@PaulOgilvie Excel addins use byte-counted not null-terminated strings, hence the space at the start of each string literal in functionTable. The length of the string is set in the first byte. — DS_London, Apr 07 '21 at 07:12
This is interesting. I have tried in the past to code a more generic function registration mechanism and run into this kind of issue (64 bit VS) ... which to be honest I never solved, just worked around. I also found that code that worked as a debug build, crashed in release versions: this may be because debug is add guard bytes around the memory, perhaps? — DS_London, Apr 07 '21 at 07:21

Adrian Mole · Accepted Answer · 2021-04-01T15:05:53.567

Although I don't (yet) have a full explanation for this behaviour, I'm posting this as a possible 'workaround', which I have used in a very similar case I encountered in one of my projects.

The issue appears to be some form of 'stack corruption' caused by the use of the function-local variable (i) used as the loop index; converting this to a global/static variable will likely fix the issue. The following code snippet is a suggested fix (I have changed the name of the index variable to avoid possible name clashes elsewhere in the code):

///...
    static int regloop; // Used as the loop index, below...

    __declspec(dllexport) int __stdcall xlAutoOpen(){
        LPXLOPER pxDLL;
        Excel4(xlGetName,pxDLL,0);

        XLOPER xlRegArgs[11];
        for(regloop = 0; regloop < 11; regloop++){
            xlRegArgs[regloop] = *TempStr(functionTable[regloop]);
        }

Here's the section of code from my aforementioned project (but note this is C++/MFC) that exhibits the same sort of behaviour – but only in x86 builds (x64 builds work without issue):

static int plin;    // NOTA BENE:-  We use this in the two functions below, as the use of
                    // a local 'plin' loop index is prone to induce stack corruption (?), 
                    // especially in MSVC 2017 (MFC 14) builds for x86.

void BasicApp::OnUpdatePICmd(uint32_t nID, void *pUI)
{
//! for (int plin = 0; plin < Plugin_Number; ++plin) { // Can cause problems - vide supra
    for (plin = 0;  plin < Plugin_Number;  ++plin) {
        BOOL mEbl = FALSE;  int mChk = -1;
        if ((Plugin_UDCfnc[plin] != nullptr) && Plugin_UDCfnc[plin](nID, &mEbl, &mChk)) {
            CommandEnable(pUI, mEbl ? true : false);
            if (mChk >= 0) CmdUISetCheck(pUI, mChk);
            return;
        }
    }
    CommandEnable(pUI, false);
    return;
}

(The Plugin_UDCfnc is a static array member of the BasicApp class.)

I have, in the years since the above code was written, had occasional 'fleeting insights' into why this is happening but, as of now, I can't offer a more robust fix. I shall revisit the issue and update this post if I should stumble upon a resolution. In the meantime, others are welcome to take this as a 'clue' and post their own explanations/solutions.

I've installed x64 version Excel in a virtual machine, copyed the corresponding x64 XLCALL32.dll to the source directory, and compiled the original local iterator `i` version source file: `x86_64-w64-mingw32-gcc -shared -Wl,--kill-at testXLL.c -o win64.xll -L. -lXLCALL32`. This time x64' xll works fine, without crash like x86. Indeed, maybe this issue is exclusive for x86. — user3148104, Apr 01 '21 at 14:37

Excel Add-in: Assignment in for loop causes segmentation fault but line-by-line assignments work. Why?

1 Answers1