PHP pthreads async with threads number gets stuck

Question

I'm trying to achive a script that start threads async with a limit(5) and does wait if all threads are busy. If the threads are busy my script has to wait untill one is free and it should start another one but for some reasons does get stuck. Yesterday I discovered pthreads and after a few hours today this is all I came with:

EDIT: There are examples with 500 threads for example but in my case I need to parse large files(a few MBs) of targets and I need to make it to work async.

<?php
class My extends Thread {
  public function run() {
    echo $this->getThreadId() . "\n";
    sleep(rand(1, 3)); // Simulate some work...
    global $active_threads;
    $active_threads = $active_threads - 1; // It should decrese the number of active threads. Here could be the problem?
  }
}

$active_threads = 0;
$maximum_threads = 5;

while(true) {
  if($active_threads == $maximum_threads) {
    echo 'Too many threads, just wait for a moment untill some threads are free.' . "\n";
    sleep(3);
    continue;
  }

  $threads = array();
  for($i = $active_threads; $i < $maximum_threads; $i++) {
    $active_threads = $active_threads + 1;
    $threads[] = new My();
  }

  foreach($threads as $thread) {
    $thread->start();
  }
}
?>

Output:

[root@localhost tests]# zts-php test
140517320455936
140517237061376
140517228668672
140517220275968
140517207701248
Too many threads, just wait for a moment.
Too many threads, just wait for a moment.
Too many threads, just wait for a moment.
Too many threads, just wait for a moment.

I know is pretty primitive but I'm totally new to pthreads and all the examples are without async threads number limit. There is for sure something wrong in my code but I have no ideea where.

What you want is a pool of workers. Check out the pool class documentation of pthreads. — Charlotte Dunois, Aug 22 '19 at 20:53

score 0 · Answer 1 · answered Aug 22 '19 at 20:10

0

The problem as stated in another post is that the global memory is not shared. So your $active_thread will not decrement even if you use global.

See post: How to share global variable across thread in php?

answered Aug 22 '19 at 20:10

jasonwubz

341
3
6

There is the problem for sure but I was thinking that would be fixed by pthreads, isn't that the point of pthreads? – John Doe Aug 23 '19 at 05:37
@JohnDoe Well, that is not the case right now. But you can do a workaround it by using memcache or redis to store manage your counter variable. I am certain that with redis you can handle the atomicity part so to avoid any race conditions with the counter. Hope it helps! – jasonwubz Aug 23 '19 at 13:14
Well is already getting complicated using other stuffs, I already manager after a few hours of tinkering and testing to write another script from scratch and is working and is doing what I wanted too. – John Doe Aug 23 '19 at 14:01

PHP pthreads async with threads number gets stuck

1 Answers1