Returning a multidimensional array from C to Lua

Question

My question is admittedly similar to this: How to create nested Lua tables using the C API

I thought that I understood that answer but I continue to have issues.

I have an array of objects that I'm trying to return.

        const char * pushlist[] = {
              "status", "cmdsequence", "timestamp", "gain",
        };
        int nItems = sizeof(pushlist) / sizeof(char *);
        int iDepth = -(1 + nItems);

        // //
        // What we want to do is essentially push an array of tables.
        // The tables have keys (see pushlist above) and values.
        // The array is indexed by integers from 1 through N.
        //
        lua_newtable( L );
        for( Json::Value::UInt i = 0; i != totFrames; i++ )
        {
            lua_pushnumber( L, i + 1 );  // push the array index
            lua_newtable( L );

            Json::Value frame = params["frameinfo"][i];

            // now push the table which will be at array index (i + 1)
            for( int n = 0; n < nItems; n++ )
            {
                lua_pushstring( L, pushlist[n] );                 // push key
                lua_pushnumber( L, frame[pushlist[n]].asUInt() ); // push value
            }
            lua_settable(L, iDepth);
            lua_settable(L, -3);    // (note 1) error here
        }
        lua_settable(L, iDepth);    // (note 2) not clear on the need for this
        lua_settable(L, -3);
        lua_setglobal( L, "framedata" );

So in Lua I want to see:
[0] = {["status"] = 1, ["cmdsequence"] = 2, ["timestamp"] = 3, ["gain"] = 4}
...
[totFrames-1] = {["status"] = 5, ["cmdsequence"] = 6, ["timestamp"] = 7, ["gain"] = 8}

I'm not clear on what the 2 lua_settable's are for at note 2, but the answer I linked to above indicates they're needed.

The lua_settable(L, -3) (note 1) errors out. I'm doing this in C++ and so I have that code bracketed in a try/catch. When it hits that settable the first time it bails and goes to my catch. I'm thinking I've corrupted the stack somehow but I don't see it.

---

Thanks @Omri Barel for the excellent answer. I'm still not clear on what to do after the inner 'for' loop.

I now have this: const char * pushlist[] = { "status", "cmdsequence", "timestamp", "gain", }; int nItems = sizeof(pushlist) / sizeof(char *);

        // //
        // What we want to do is essentially push an array of tables.
        // The tables have keys (see pushlist above) and values.
        // The array is indexed by integers from 1 through N.
        //
        lua_newtable( L );
        for( Json::Value::UInt i = 0; i != totFrames; i++ )
        {
            lua_pushnumber( L, i + 1 );  // push the array index
            lua_newtable( L );

            Json::Value frame = params["frameinfo"][i];

            // now push the table which will be at array index (i + 1)
            for( int n = 0; n < nItems; n++ )
            {
                lua_pushnumber( L, frame[pushlist[n]].asDouble() ); // push value
                lua_setfield(L, -2, pushlist[n] );
            }
            lua_settable(L, -3);    // (note 1) error here
        }
        //lua_settable(L, -3);       <<-- not certain that this is required
        lua_setglobal( L, "framedata" );

I no longer blow up, but my Lua fails (no error message, it simply exits). I suspect that I haven't corrupted the stack, but somehow I haven't completed this table properly and so my return is confused.

I push several other return values into the Lua stack before this array, and then one more after it.

My Lua call is like this:

       param1,param2,framedata,Err = CCall.ReadFromC( arg, arg );

---

I finally have this working. It needs further testing, but seems correct so far. Thanks again to @Omri Barel. Here's the code snippet that I've ended up with.

        const char * pushlist[] = {
              "status", "cmdsequence", "timestamp", "gain",
        };
        int nItems = sizeof(pushlist) / sizeof(char *);

        // //
        // What we want to do is essentially push an array of tables.
        // The tables have keys (see pushlist above) and values.
        // The array is indexed by integers from 1 through N.
        //
        lua_newtable( L );
        for( Json::Value::UInt i = 0; i != totFrames; i++ )
        {
            Json::Value frame = params["frameinfo"][i];

            // now push the table which will be at array index (i + 1)
            lua_newtable( L );
            for( int n = 0; n < nItems; n++ )
            {
                const char * itemName = pushlist[n];
                if( frame[itemName].isNull() ) continue;
                lua_pushnumber( L, frame[pushlist[n]].asDouble() ); // push value
                lua_setfield(L, -2, pushlist[n] );
            }
            lua_rawseti(L, -2, i + 1);
        }

Answer 1

The problem is that you are pushing too many key/value pairs before you add them to the table. You should add one pair at a time.

When debugging Lua, the best thing you can do is dump the content of the stack very often, to check what's going on. There's some "stack dumping" code in "Programming In Lua" (first edition is good enough for this).

Let's have a look at your code (inside the frame loop).

1. You create a table. The stack is:

   ... [table]

2. You push pairs of key / value:

   ... [table] [key] [value] [key] [value] [key] [value] [key] [value]

3. You call settable with iDepth, which doesn't look right to me. In this case iDepth = -(1+nItems) = -5. But you are pushing pairs, so it should be double that. But even if iDepth were right, you're still only calling it once, so it only removes a single pair from the stack:

   ... [table] [key] [value] [key] [value] [key] [value]

4. You call settable with -3, but -3 is [value] so you get an error.

You should call settable (with -3) after each key / value pair.

And I would also suggest using setfield which is a bit clearer. Instead of:

lua_pushstring(L, key);
lua_pushnumber(L, value);
lua_settable(L, -3);

You can use:

lua_pushnumber(L, value);
lua_setfield(L, -2, "key");