0

I'm trying to open 20 concurrent tabs in chrome browser using php and chrome-php library.
Each tab navigates to the URL, evaluates some js code, and gets the data from the page.

foreach ($urls as $url) {
    $page = $this->browser->createPage();

    $page->navigate($url)->waitForNavigation();
    $page->close();
}

If I'm trying to use Spatie\Async\Pool

$pool = Pool::create();

foreach ($urls as $url) {
    $pool[] = async(function () use ($url) {
      $page = $this->browser->createPage();

      $page->navigate($url)->waitForNavigation();
      // evaluate js code
      $page->close();
    });
}

await($pool);

I get this error:
Cannot assign Laravel\SerializableClosure\Serializers\Native to property Symfony\Component\Console\Input\InputArgument::$suggestedValues of type Closure|array

Running sequentially will take a lot of time, is that possible to achieve concurrency?

crafter
  • 54
  • 7
  • I have no idea about your specific error message, however Spatie\Async\Pool isn't intended for a "multiple tab" situation, it is intended for take a single process (one tab, if you will) and break the code up to run async. So if you had 100 API calls on page load, you could pool those for the current request and let them complete on their own schedule. But you still have a dedicate synchronous "mail process" for each request. – Chris Haas Dec 21 '22 at 14:29
  • @ChrisHaas, thanks for the reply. `Spatie\Async\Pool` is just an example of what I did and trying to achieve. – crafter Dec 21 '22 at 15:04
  • 1
    The general pattern you are looking for would be called a queue, however you'll need to add some custom UI on top of it, and I'm not sure what all Laravel has support for out of the box. The general idea is that an HTTP request comes in and instead of processing it on the spot, it gets add to the queue and a message is returned to the user, probably with some form of "ticket" to check on the queue status. Then there's a task runner on the server the monitors the queue, runs the tasks and somehow marks them as complete. – Chris Haas Dec 21 '22 at 17:07
  • You can use libraries and special-built binaries for all of this, or you could just have a normal entity with a "status" column with values of "done", "processing", "ready-to-process", etc., and then have a background cron or daemon that's just PHP (a Laravel CLI app ideally) that process things. – Chris Haas Dec 21 '22 at 17:08
  • 1
    Laravel does support queues and technically it may work. On nodejs for an instance, I can use es6-promise-pool to use concurrency with promises, well because of the nature of js. That's the reason I was thinking about how to do that in PHP. – crafter Dec 21 '22 at 18:04
  • The major difference between nodejs and PHP (and thus Laravel) is that PHP is almost always run behind a web server such as Apache, Nginx or IIS, and _those_ systems isolate each request to a dedicated process (or equivalent) which means that async is still only-per request. That said, projects such as [Swoole](https://openswoole.com/) are trying to fix/change this, and they are to a certain degree evolving PHP itself through [fibers](https://www.php.net/manual/en/language.fibers.php) to behave in a more async fashion. You might want to investigate that if this is something you really want. – Chris Haas Dec 21 '22 at 18:53

0 Answers0