luabind: cannot retrieve values from table indexed by non-built-in classes‏

Question

I'm using luabind 0.9.1 from Ryan Pavlik's master distribution with Lua 5.1, cygwin on Win XP SP3 + latest patches x86, boost 1.48, gcc 4.3.4. Lua and boost are cygwin pre-compiled versions.

I've successfully built luabind in both static and shared versions.

Both versions pass all the tests EXCEPT for the test_object_identity.cpp test which fails in both versions.

I've tracked down the problem to the following issue: If an entry in a table is created for NON built-in class (i.e., not int, string, etc), the value CANNOT be retrieved.

Here's a code piece that demonstrates this:

#include "test.hpp"
#include <luabind/luabind.hpp>
#include <luabind/detail/debug.hpp>

using namespace luabind;

struct test_param
{
    int obj;
};

void test_main(lua_State* L)
{
    using namespace luabind;

    module(L)
    [
        class_<test_param>("test_param")
            .def_readwrite("obj", &test_param::obj)
    ];

    test_param temp_object;
    object tabc = newtable(L);
    tabc[1] = 10;
    tabc[temp_object] = 30;

    TEST_CHECK( tabc[1] == 10 );              // passes
    TEST_CHECK( tabc[temp_object] == 30 );    // FAILS!!!

}

tabc[1] is indeed 10 while tabc[temp_object] is NOT 30! (actually, it seems to be nil)

However, if I use iterate to go over tabc entries, there're the two entries with the CORRECT key/value pairs.

Any ideas?

BTW, overloading the == operator like this:

#include <luabind/operator.hpp>

struct test_param
{
    int obj;
    bool operator==(test_param const& rhs) const
    {
        return obj == rhs.obj;
    }
};

and

module(L)
    [
        class_<test_param>("test_param")
            .def_readwrite("obj", &test_param::obj)
            .def(const_self == const_self)
    ];

Doesn't change the result.

I also tried switching to settable() and gettable() from the [] operator. The result is the same. I can see with the debugger that default conversion of the key is invoked, so I guess the error arises from somewhere therein, but it's beyond me to figure out what exactly the problem is.

As the following simple test case show, there're definitely a bug in Luabind's conversion for complex types:

struct test_param : wrap_base 
{ 
    int obj; 
    bool operator==(test_param const& rhs) const 
    { return obj == rhs.obj ; } 
}; 

void test_main(lua_State* L) 
{ 
    using namespace luabind; 
    module(L) 
    [ 
        class_<test_param>("test_param") 
                .def(constructor<>()) 
                .def_readwrite("obj", &test_param::obj) 
                .def(const_self == const_self) 
    ]; 

    object tabc, zzk, zzv; 
    test_param tp, tp1; 
    tp.obj = 123456; 
    // create new table 
    tabc = newtable(L); 
    // set tabc[tp] = 5; 
    //         o     k   v 
    settable( tabc,  tp, 5); 
    // get access to entry through iterator() API 
    iterator zzi(tabc); 
    // get the key object 
    zzk = zzi.key(); 
    // read back the value through gettable() API 
    //              o     k 
    zzv = gettable(tabc, zzk);   
    // check the entry has the same value 
    // irrespective of access method 
    TEST_CHECK ( *zzi == 5 && 
                 object_cast<int>(zzv) == 5 ); 
    // convert key to its REAL type (test_param) 
    tp1 = object_cast<test_param>(zzk); 
    // check two keys are the same 
    TEST_CHECK( tp == tp1 ); 
    // read the value back from table using REAL key type 
    zzv = gettable(tabc, tp1); 
    // check the value 
    TEST_CHECK( object_cast<int>(zzv) == 5 ); 
    // the previous call FAILS with 
    // Terminated with exception: "unable to make cast" 
    // this is because gettable() doesn't return 
    // a TRUE value, but nil instead 
} 

Hopefully, someone smarter than me can figure this out, Thx

I've traced the problem to the fact that Luabind creates a NEW DISTINCT object EVERY time you use a complex value as key (but it does NOT if you use a primitive one or an object).

Here's a small test case that demonstrates this:

struct test_param : wrap_base
{
    int obj;
    bool operator==(test_param const& rhs) const
    { return obj == rhs.obj ; }
};

void test_main(lua_State* L)
{
    using namespace luabind;

    module(L)
    [
        class_<test_param>("test_param")
            .def(constructor<>())
            .def_readwrite("obj", &test_param::obj)
            .def(const_self == const_self)
    ];

    object tabc, zzk, zzv;
    test_param tp;
    tp.obj = 123456;
    tabc = newtable(L);
    //         o     k   v
    settable( tabc,  tp, 5);
    iterator zzi(tabc), end;
    std::cerr << "value = " << *zzi << "\n";
    zzk = zzi.key();
    //         o     k    v
    settable( tabc,  tp,  6);
    settable( tabc,  zzk, 7);
    for (zzi = iterator(tabc); zzi != end; ++zzi)
    {
        std::cerr << "value = " << *zzi << "\n";
    }
}

Notice how tabc[tp] first has the value 5 and then is overwritten with 7 when accessed through the key object. However, when accessed AGAIN through tp, a new entry gets created. This is why gettable() fails subsequently.

Thx, David

Answer 1

Disclaimer: I'm not an expert on luabind. It's entirely possible I've missed something about luabind's capabilities.

First of all, what is luabind doing when converting test_param to a Lua key? The default policy is copy. To quote the luabind documentation:

This will make a copy of the parameter. This is the default behavior when passing parameters by-value. Note that this can only be used when passing from C++ to Lua. This policy requires that the parameter type has an accessible copy constructor.

In pratice, what this means is that luabind will create a new object (called "full userdata") which is owned by the Lua garbage collector and will copy your struct into it. This is a very safe thing to do because it no longer matters what you do with the c++ object; the Lua object will stick around without really any overhead. This is a good way to do bindings for by-value sorts of objects.

Why does luabind create a new object each time you pass it to Lua? Well, what else could it do? It doesn't matter if the address of the passed object is the same, because the original c++ object could have changed or been destroyed since it was first passed to Lua. (Remember, it was copied to Lua by value, not by reference.) So, with only ==, luabind would have to maintain a list of every object of that type which had ever been passed to Lua (possibly weakly) and compare your object against each one to see if it matches. luabind doesn't do this (nor do I think should it).

Now, let's look at the Lua side. Even though luabind creates two different objects, they're still equal, right? Well, the first problem is that, besides certain built-in types, Lua can only hold objects by reference. Each of those "full userdata" that I mentioned before is actually a pointer. That means that they are not identical.

But they are equal, if we define an __eq meta operation. Unfortunately, Lua itself simply does not support this case. Userdata when used as table keys are always compared by identity, no matter what. This actually isn't special for userdata; it is also true for tables. (Note that to properly support this case, Lua would need to override the hashcode operation on the object in addition to __eq. Lua also does not support overriding the hashcode operation.) I can't speak for the authors of Lua why they did not allow this (and it has been suggested before), but there it is.

So, what are the options?

• The simplest thing would be to convert test_param to an object once (explicitly), and then use that object to index the table both times. However, I suspect that while this fixes your toy example, it isn't very helpful in practice.
• Another option is simply not to use such types as keys. Actually, I think this is a very good suggestion, since this kind of light-weight binding is quite useful, and the only other option is to discard it.
• It looks like you can define a custom conversion on your type. In your example, it might be reasonable to convert your type to a Lua number which will behave well as a table index.
• Use a different kind of binding. There will be some overhead, but if you want identity, you'll have to live with it. It sounds like luabind has some support for wrappers, which you may need to use to preserve identity:

  When a pointer or reference to a registered class with a wrapper is passed to Lua, luabind will query for it's dynamic type. If the dynamic type inherits from wrap_base, object identity is preserved.