Can I set a custom name displayed for a gevent greenlet while logging?

Question

When I print out log statements from within greenlets using the standard logging package, I get text that looks like this:

2014-02-06 22:38:43,428 [INFO] (11396-Dummy-2) event_listener: About to block

I'm referring to the 11396-Dummy-2 portion. I'd like it instead to say something like 'Main' or 'Listener 1', for instance. Is this possible? From the current docs I don't see any API to do so.

score 5 · Answer 1 · answered Feb 07 '14 at 18:04

I'm not sure where the 11396-Dummy-2 portion in your logs is coming from, but if you want to add contextual information to the log (such as a greenlet identifier), there are a number of documented ways of doing it. Here's an example, using a LoggerAdapter:

import logging
import gevent

class Adapter(logging.LoggerAdapter):
    def process(self, msg, kwargs):
        msg = '(%s) %s' % (gevent.getcurrent()._run.__name__, msg)
        return msg, kwargs

logger = Adapter(logging.getLogger(), {})

def foo():
    logger.debug('Running in foo')
    gevent.sleep(0)
    logger.debug('Explicit context switch to foo again')

def bar():
    logger.debug('Explicit context to bar')
    gevent.sleep(0)
    logger.debug('Implicit context switch back to bar')

logging.basicConfig(level=logging.DEBUG,
                    format='%(levelname)s %(threadName)s %(message)s')
gevent.joinall([
    gevent.spawn(foo),
    gevent.spawn(bar),
])

When run, this should print

DEBUG MainThread (foo) Running in foo
DEBUG MainThread (bar) Explicit context to bar
DEBUG MainThread (foo) Explicit context switch to foo again
DEBUG MainThread (bar) Implicit context switch back to bar

This should be the accepted answer. Since it uses the logging adapter and standard gevent functionality. — Realistic, Dec 04 '15 at 15:55

score 4 · Answer 2 · edited May 23 '17 at 12:31

Note, that if threading module is monkey-patched, threads are pretty much mapped to greenlets. In particular monkey patching replaces _start_new_thread() (so it starts a new greenlet instead), as well as _get_ident() (so greenlet ID is returned whenever a thread id is being asked for). Thanks to this mapping, whenever you ask about current thread, in fact you get a dummy Thread object instance associated with currently running greenlet!

Thus, it is perfectly possible to do the following:

import gevent.monkey
gevent.monkey.patch_thread()

from threading import current_thread

# and then, at the start of the greenlet
current_thread().name = "MyNewName"

Now, whenever logging code retrieves current thread name, it gets per-greenlet name. I've to admit it's a bit of a hack, but it works pretty well in my current project.

And here's a proof of concept:

import gevent.monkey
gevent.monkey.patch_thread()

import logging
from threading import current_thread

logger = logging.getLogger()

def foo():
    current_thread().name = "MyFoo" 
    logger.debug('Running in foo')
    gevent.sleep(0)
    logger.debug('Explicit context switch to foo again')

def bar():
    current_thread().name = "MyBar" 
    logger.debug('Explicit context to bar')
    gevent.sleep(0)
    logger.debug('Implicit context switch back to bar')

logging.basicConfig(level=logging.DEBUG,
                    format='%(levelname)s %(threadName)s %(message)s')
gevent.joinall([
    gevent.spawn(foo),
    gevent.spawn(bar),
])

When executed, it prints:

DEBUG MyFoo Running in foo
DEBUG MyBar Explicit context to bar
DEBUG MyFoo Explicit context switch to foo again
DEBUG MyBar Implicit context switch back to bar

Just ensure that threading module is patched before any other import (see this answer).

Can I set a custom name displayed for a gevent greenlet while logging?

2 Answers2