0
-- I launch 10 threads
-- Threads share Queue.Queue().
-- Each Thread gets an item from the Queue

    -- Each thread communicates first with External WebServer
    -- Processes Data coming from Ext. WebServer
    -- Communicates with internal DB
    -- Gets the next the item from Queue
    -- It repeats until Queue finishes
    -- At which point all threads are joined and main thread exits normally

    -- In the code, in main thread I populate Queue.Queue() with 500 items
    -- Then each thread gets the first item in the queue.

I observe that:

-- Initially (that means when each thread is processing its first item) all 10 threads starts and proceeds normally -- However, second item onwards, only 2 from 10 threads progress. -- Since there are multiple network, I/O operations involved, I assume all threads should interleave and get some CPU time.

Could you please explain the behavior? Any pointers to documentation or previously asked question on stack would be helpful.

sysctl hw.ncpu (mac OS) hw.ncpu: 2

Regards,

mechanical_meat
  • 163,903
  • 24
  • 228
  • 223
gaurav jain
  • 1,267
  • 8
  • 20
  • 36
  • Do you have an example on how to reproduce this? – Emil Vikström Jun 04 '12 at 16:37
  • Let me work on it. Give me some time. Although the code I will provide will need to be run at mac OS with nCPU = 2 – gaurav jain Jun 04 '12 at 16:39
  • What happens if you run with only 1 thread. If that runs to completion, try 2. Then more... – sizzzzlerz Jun 04 '12 at 16:46
  • Configured with 1 and 2 threads, it runs as expected. I wanted to try 3, 4, ... before asking the question on stack. But I asked anyways in a hope if somebody has the answer already. – gaurav jain Jun 04 '12 at 16:49
  • hmm. Insteading of talking to ext. server, if I put a sleep and have 20 threads, all works. There is some more investigation I need to do. Probably defer this question for now. Apologies!! – gaurav jain Jun 04 '12 at 18:27
  • Please check GIL in python: https://stackoverflow.com/questions/10568358/jython-multithreading – Om Prasad Nayak Dec 18 '20 at 07:15

0 Answers0