uuid.uuid1, uuid_generate_time, and threads

Question

In Python, I can consistently generate a seg fault in the uuid module. This can be done by calling uuid.uuid1() repeatedly from multiple threads. After some digging, it appears that this function eventually calls the C uuid_generate_time function via ctypes:

From uuid.py:

for libname in ['uuid', 'c']:
    try:
        lib = ctypes.CDLL(ctypes.util.find_library(libname))
    except:
        continue
    if hasattr(lib, 'uuid_generate_random'):
        _uuid_generate_random = lib.uuid_generate_random
    if hasattr(lib, 'uuid_generate_time'):
        _uuid_generate_time = lib.uuid_generate_time
        if _uuid_generate_random is not None:
            break  # found everything we were looking for

And later on in the definition of uuid1():

def uuid1(node=None, clock_seq=None):
    """Generate a UUID from a host ID, sequence number, and the current time.
    If 'node' is not given, getnode() is used to obtain the hardware
    address.  If 'clock_seq' is given, it is used as the sequence number;
    otherwise a random 14-bit sequence number is chosen."""

    # When the system provides a version-1 UUID generator, use it (but don't
    # use UuidCreate here because its UUIDs don't conform to RFC 4122).
    if _uuid_generate_time and node is clock_seq is None:
        _buffer = ctypes.create_string_buffer(16)
        _uuid_generate_time(_buffer)
        return UUID(bytes=_buffer.raw)

I've read the man pages for uuid_generate_time as well as the Python docs for uuid.uuid1 and there is no mention of thread safety. I assume it has something to do with the fact that it needs access the system clock and/or MAC address but that's just a blind guess.

I was wondering if anyone could enlighten me?

The following is the code that I used to generate the seg fault:

import threading, uuid

# XXX If I use a lock, I can avoid the seg fault
#uuid_lock = threading.Lock()
def test_uuid(test_func):
  for i in xrange(100):
    test_func()
    #with uuid_lock:
    #  test_func()

def test(test_func, threads):
  print 'Running %s with %s threads...' % (test_func.__name__, threads)

  workers = [threading.Thread(target=test_uuid, args=(test_func,)) for x in xrange(threads)]
  [x.start() for x in workers]
  [x.join() for x in workers]
  print 'Done!'

if __name__ == '__main__':
  test(uuid.uuid4, 8)
  test(uuid.uuid1, 8)

The output that I get is:

Running uuid4 with 8 threads...
Done!
Running uuid1 with 8 threads...
Segmentation Fault (core dumped)

Oh, and I'm running this on Solaris...

For what it's worth, I ran this a bunch of times on python 2.7.8 on os x 10.9.4 and even increased the threads to 100, and did not get a segfault. This may be a platform-specific issue. — Jason S, Jul 29 '14 at 02:27
@JasonS yes I was afraid of that, this was on Solaris; I'll make a note of that in the question. Thanks! — beetea, Jul 29 '14 at 02:30
There is a comment in cpython code saying that is not thread safe: https://github.com/python/cpython/blob/master/Lib/uuid.py#L481 — Gonzalo Matheu, Apr 18 '17 at 00:15
We had a similar issue, but instead of seg fault, we reached open files limit. I.e: https://github.com/dln/python-libuuid/issues/1 — Gonzalo Matheu, Apr 18 '17 at 00:17

score 0 · Answer 1 · answered Jul 29 '14 at 04:46

The documentation doesn't say it's thread safe so you can't assume it is. It's as simple as that.

Looking at the current OpenIndiana source for uuid_generate_time it's not entirely obvious what's causing the segfault. However while the function does use a lock, it doesn't hold this lock while performing a number of initialization tasks. This is probably related to the problem, but I can't point to a specific spot where a race condition would cause a fault. You might try calling uuid1 once before you start any threads and see if that makes the problem go away. Though you'd be better off just using your own lock, because there's no guarantee that the Python code itself is thread safe.

uuid.uuid1, uuid_generate_time, and threads

1 Answers1