5

I have a python project that relies on some lua files, one of which requires 'socket'. I get an error loading socket.core "undefined symbol: lua_getmetatable" when I try to require that lua file from python 2.7.

Simple reproducer:

$ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lua
>>> lua.require('socket')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: error: error loading module 'socket.core' from file 
    '/usr/lib/lua/5.1/socket/core.so':
    /usr/lib/lua/5.1/socket/core.so: undefined symbol: lua_getmetatable

I am using a recent Lunatic Python branch where I cleaned up the Py_ssize_t warnings, and liblua5.1-socket2 on Ubuntu 11.04

I get the same error if I use the main lunatic-python source code and/or upgrade to luasocket 2.0.2.

edit: What is causing this error, and how do I fix it?

edit #2: Here is the output of building luasocket-2.0.2. The default make did not build unix.so, and I edited it to build that as well so I didn't mix and match 2.0.0 with 2.0.2:

$ make
cd src; make all
make[1]: Entering directory `/sandbox/luasocket/luasocket-2.0.2/src'
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o luasocket.o luasocket.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o timeout.o timeout.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o buffer.o buffer.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o io.o io.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o auxiliar.o auxiliar.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o options.o options.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o inet.o inet.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o tcp.o tcp.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o udp.o udp.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o except.o except.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o select.o select.c
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o usocket.o usocket.c
gcc  -O -shared -fpic -o socket.so.2.0.2 luasocket.o timeout.o buffer.o io.o auxiliar.o options.o inet.o tcp.o udp.o except.o select.o usocket.o 
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o mime.o mime.c
gcc  -O -shared -fpic -o mime.so.1.0.2 mime.o 
gcc -I/usr/include/lua5.1 -DLUASOCKET_DEBUG  -pedantic -Wall -O2 -fpic -llua   -c -o unix.o unix.c
gcc  -O -shared -fpic -o unix.so buffer.o auxiliar.o options.o timeout.o io.o usocket.o unix.o
make[1]: Leaving directory `/sandbox/luasocket/luasocket-2.0.2/src'
RedCup
  • 63
  • 6
  • What is the question, though? ;) – 0xC0000022L Dec 02 '11 at 19:23
  • 1
    What is causing this error, and how do I fix it? – RedCup Dec 02 '11 at 20:28
  • The symbols are not correctly exported from the `lua` module that could be used in module it requires. Could you also post a log of the compilation stage, mainly the final linking part? – Michal Kottman Dec 02 '11 at 20:47
  • @MichalKottman Added compile and link output to the original question. – RedCup Dec 02 '11 at 21:38
  • have you tried to run `ldconfig`? if `/usr/lib/lua/5.1` is a symbolic link then `echo "/path/to/lue5.1/lib" > /etc/ld.so.conf.d/lua51.conf && ldconfig` – jfs Dec 03 '11 at 22:24
  • /usr/lib/lua/5.1/socket/core.so is a symlink to /usr/lib/liblua5.1-socket.so.2.0.0 Adding /usr/lib or even /usr/lib/lua/5.1/socket as suggested by @JFSebastian (with elevated privs) has no affect on the error. I think it's finding the right so, as the error complains about /usr/lib/lua/5.1/socket/core.so – RedCup Dec 05 '11 at 23:11

1 Answers1

6

The problem is not in luasocket, but in the way symbols from shared libraries are handled.

The problem is that the while the lua.so (Python module) links to liblua5.1.so, shared modules loaded by require do not have access to the symbols from liblua5.1.so. On Mac OS X this works, because symbols from dlopen are loaded as RTLD_GLOBAL by default.

I have experimented with modifying the Lua source (lua-5.1.4/src/loadlib.c:69), however it does not help, because by the time require is called from lua.so, the symbols from liblua5.1.so have already been loaded locally for lua.so. This is why luasocket does not see them.

Fortunately, Python allows you to change the dlopen semantics using the sys module. This allows you to force to load modules with RTLD_GLOBAL, which is exactly what is needed. Try running the following code and see if it works for you:

$ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, DLFCN
>>> sys.setdlopenflags(DLFCN.RTLD_NOW | DLFCN.RTLD_GLOBAL)
>>> import lua
>>> lua.require("socket")
<Lua table at 0x22ccef0>
Michal Kottman
  • 16,375
  • 3
  • 47
  • 62
  • Wow... Thanks for the great explanation too. For the curious, you can learn more about ldload via 'man ldload' and the [Python docs for the sys module](http://docs.python.org/library/sys.html). sys.setdlopenflags( n ) is unix only. – RedCup Dec 06 '11 at 18:01
  • I'm running into a similar problem except on the lua side instead of python. I tried rebuilding lunatic with `-export-dynamic` hoping that symbols it depends on would be visible to subsequent modules loaded later -- it didn't work. Out of curiosity, why doesn't this problem happen on windows? – greatwolf Apr 15 '13 at 23:52