2

I'm new to both Python and multi-threading. I read that a Python thread finishes when the function it is running 'returns'. Should I be worried that my program is spinning up an infinite amount of threads?

I'm asking because sometimes the HTTP request times out and it throws an (expected) error:

Exception in Thread-44

The error is not my worry, but rather the thread number is. Does this means there are indeed 44 threads or that this is simply the 44th thread (but the previous 43 or so closed gracefully)?

Code:

def start(self, error_refresh=None):
  self.timeout_id = GLib.timeout_add_seconds(10, self.check_price)

def stop(self):
  if self.timeout_id not 0:
    GLib.source_remove(self.timeout_id)

def check_price(self):
  self.async_get(URL, callback=self._parse_result)
  return True

def async_get(self, *args, callback=None, timeout=15, **kwargs):
  if callback:
      def callback_with_args(response, *args, **kwargs):
          callback(response)
      kwargs['hooks'] = {'response': callback_with_args}
  kwargs['timeout'] = timeout
  thread = Thread(target=requests.get, args=args, kwargs=kwargs)
  thread.start()

def _parse_result(self, data):
  ## code for parsing here
  GLib.idle_add(self.indicator.set_data,label, bid, high, low, ask, vol)
martineau
  • 119,623
  • 25
  • 170
  • 301
bluppfisk
  • 2,538
  • 3
  • 27
  • 56

1 Answers1

1

Hmm, let's approach the question from a slightly different angle.

Simply put, Thread pools allow you to allocate a pool with a max # of workers for requests. This means you can specify an upper bound on the # of threads being used for asynchronous tasks. Hence your concern could partially be mitigated by slightly changing how you create/run threads.

I would recommend looking into thread pools as the next step in learning threads.

Here's a useful link:

Threading pool similar to the multiprocessing Pool?

As to whether-or-not you have 43+ threads running concurrently, I don't know that we have enough information regarding your code. You'll need to do some debugging to assess how many threads are active at a given time, and if any which encountered exceptions are still present in memory, holding onto resources, or being held by other resources. IIRC, thread ID #s may be recycled.

bedwyr
  • 5,774
  • 4
  • 31
  • 49
  • Thanks, this will be useful for future optimisation of the code. So far I've found a function that tells you how many threads are running. And while there are only supposed to be 4 at a time, sometimes that number is exceeded when a thread hasn't reached its timeout yet but the timer already spun up another one. I could set the timeout to the refresh interval too. At any rate I never had more than 8, so that means the threads are correctly terminated and the number is just an ID but does not indicate the number of running threads. – bluppfisk Dec 16 '17 at 03:35
  • Most excellent! Glad you were able to find a way to ID/count your threads. Hats off to you for finding a solution :) – bedwyr Dec 18 '17 at 15:39
  • :D Also, for completeness, the function is threading.active_count() – bluppfisk Dec 18 '17 at 18:09