3

In my gevent-based program, I've got a thread somewhere which is suck in a loop something like:

while True:
    gevent.sleep(0)

How can I figure out which thread this is? Is it possible to list (and get stack traces for) the running threads?

David Wolever
  • 148,955
  • 89
  • 346
  • 502
  • Stack traces for running threads? Do you mean running greenlets? If so: http://stackoverflow.com/questions/12510648/in-gevent-how-can-i-dump-stack-traces-of-all-running-greenlets – kkurian Nov 17 '13 at 04:27
  • There is no threading in gevent, You should try to set a timeout for those gevent.spawn – Zhuo.M Jun 17 '14 at 01:26
  • … hu? gevent green threads are called threads - they are just user space threads as opposed to OS threads. – David Wolever Jun 17 '14 at 01:28

2 Answers2

1

method 1. TimeOut

I use this in my code to keep track greenlets that potentially block. A NodeTaskTimeout is raised when this happens. Just wrap your jobs in a Timeout or provide them with a TimeOut object.

 with Timeout(90, False):
        task_jobs.join()
    
    if task_jobs:
        print 'task jobs killed', task_jobs
        task_jobs.kill()
        if settings.DEBUG:
            raise NodeTaskTimeout

This print's out the task if it hangs/blocks/takes to long. Especially nasty are those jobs that depend on each other and cause a deadlock job1 /thread -> job2/thread2 -> job3/thread3 and job/thread3 only finishes when job1 is done wich will never happen because job2 is not done and job2 is not done because of job3 is not done.. you get the idea ;)

Mehtod 2 Settrace

http://www.rfk.id.au/blog/entry/detect-gevent-blocking-with-greenlet-settrace/

but you also need to put code that you suspect to be "spinning" in a with block.

Community
  • 1
  • 1
Stephan
  • 3,679
  • 3
  • 25
  • 42
  • Sorry, but I'm not sure how this addresses the question as it assumes that I'm manually keeping track of which threads have been started. – David Wolever Sep 04 '14 at 17:45
  • Well you wrap your jobs with a TimeOut. to make sure none of them is "spinning". There are other methods to detect if a green let blocks – Stephan Sep 05 '14 at 10:09
0

There is an official API for checking the block, it will pinpoint the exact line which caused the block, check code example below:

# gevent        1.3.7
# greenlet      0.4.15
# zope.event    4.4
import gevent
from gevent import config, get_hub
from gevent.events import IEventLoopBlocked
import logging
from pprint import pformat
import time
import zope.event

# initial logging
logging.basicConfig(level=logging.INFO)
log = logging.getLogger(__name__)

# setup gevent config
# enable the monitor thread
config.monitor_thread = True
config.max_blocking_time = 4

# start the monitor thread
hub = get_hub()
monitor = hub.start_periodic_monitoring_thread()

# register the event to logging system
def g(event):
    log.error('Greenlet: {}, exceed the max blocking time: {}'.format(event.greenlet, event.blocking_time))
    log.error(pformat(event.info))
event = IEventLoopBlocked()
zope.event.subscribers.append(g)

# you can also create you own monitoring function
# def check(hub):
#     print('< periodic check in monitoring thread >')
#
# monitor.add_monitoring_function(check, 1)

def gl1():
    # use time.sleep to trigger block
    log.info('block at gl1 for 2 seconds')
    time.sleep(2)
    log.info('leave gl1 now')

def gl2():
    # use time.sleep to trigger block
    log.info('block at gl2 for 6 seconds should be detected')
    time.sleep(6)
    log.info('leave gl2 now')

def gl3():
    # gevent.sleep will not block
    log.info('gl3 will not block since it use gevent.sleep')
    gevent.sleep(8)
    log.info('leave gl3 now')


gevent.joinall([
    gevent.spawn(gl3),
    gevent.spawn(gl1),
    gevent.spawn(gl2),
])

I hope it helps!

panda912
  • 196
  • 1
  • 6