3

I'm building an integration that communicates data to several different systems via API (REST). I need to process data as quickly as possible. This is a basic layout:

  1. Parse and process data (probably into an array as below)

$data = array( Title => "Title", Subtitle => "Test", .....

  1. Submit data into service (1) $result1 = $class1->functionservice1($data);
  2. Submit data into service (2) $result2 = $class2->functionservice2($data);
  3. Submit data into service (3) $result3 = $class3->functionservice3($data);
  4. Report completion echo "done";

Run in a script as above I'll need to wait for each function to finish before it starts the next one (taking 3 times longer).

Is there an easy way to run each service function asynchronously but wait for all to complete before (5) reporting completion. I need to be able to extract data from each $result and return that as one post to a 4th service.

Sorry if this is an easy question - I'm a PHP novice

Many thanks, Ben

Ben Rees
  • 123
  • 1
  • 9
  • you need to use pthreads extension, PHP does not support threading out of the box... see [this](http://stackoverflow.com/questions/70855/how-can-one-use-multi-threading-in-php-applications) – n00dl3 Dec 19 '16 at 10:11
  • You can see here the answer for your question http://stackoverflow.com/questions/962915/how-do-i-make-an-asynchronous-get-request-in-php – Avihay m Dec 19 '16 at 10:15
  • @Avihaym there is no asynchronicity in those answers... – n00dl3 Dec 19 '16 at 10:31
  • Are you running your code from `CLI` or is your script invoked through web server? – Mjh Dec 19 '16 at 10:47
  • @Mjh - I'm running it through a web server, probably started as a cron job. I'm going to iterate over a database of pending data updates - extract one line at a time and run it through the above process. – Ben Rees Dec 19 '16 at 11:36

3 Answers3

2

Yes, there are multiple ways.

The most efficient is to use an event loop that leverages non-blocking I/O to achieve concurrency and cooperative multitasking.

One such event loop implementation is Amp. There's an HTTP client that works with Amp, it's called Artax. An example is included in its README. You should have a look at how promises and coroutines work. There's Amp\wait to mix synchronous code with async code.

<?php

Amp\run(function() {
    $client = new Amp\Artax\Client;

    // Dispatch two requests at the same time
    $promises = $client->requestMulti([
        'http://www.google.com',
        'http://www.bing.com',
    ]);

    try {
        // Yield control until all requests finish
        list($google, $bing) = (yield Amp\all($promises));
        var_dump($google->getStatus(), $bing->getStatus());
    } catch (Exception $e) {
        echo $e;
    }
});

Other ways include using threads and or processes to achieve concurrency. Using multiple processes is the easiest way if you want to use your current code. However, spawning processes isn't cheap and using threads in PHP isn't really a good thing to do.

kelunik
  • 6,750
  • 2
  • 41
  • 70
1

You can also put your code in another php file and call it using this :

 exec("nohup /usr/bin/php -f your script > /dev/null 2>&1 &");
Avihay m
  • 551
  • 1
  • 5
  • 22
  • Thanks. I had considered this approach but it didn't seem I could pass my variables into the script. Is there any way I could pass my $data array into a script called with `exec(...);` ? – Ben Rees Dec 19 '16 at 10:25
  • yes you could like this : exec("nohup /usr/bin/php -f / your script ".$var1." ".$var2." > /dev/null 2>&1 &") then get it by $argv [1]... – Avihay m Dec 19 '16 at 10:30
  • Kind of yes @Avihay m . I can create another php file and pass previously created variables into it using the above options. What I still dont understand is how to retrieve a result in my parent process after the child process has completed (i need to make sure I wait for completion as well). Is there a way of doing this? As in the question - I'm doing my best but PHP is very new to me. I've got 3 answers here, all which do the job. Yours is going to be the simplest to execute which is basically what I asked for above (the others are presumably better but im struggling to understand) – Ben Rees Dec 20 '16 at 16:25
  • @BenRees Be sure to escape the variables you're passing to the script, otherwise it might execute things you don't want. If you have any questions regarding my own answer or want to chat about PHP in general, there's a [chat room for PHP](http://chat.stackoverflow.com/rooms/11/php) you can ping me in. – kelunik Dec 20 '16 at 18:43
  • @BenRees I don`t think there is a way to get the result, the script is running and not related to his parent. I think the best result for you is to save the result in DB or move to Nodejs like i did. Good luck :) – Avihay m Dec 21 '16 at 07:37
0

If you want to use asynchronicity like you can do in other languages ie. using threads, you will need to install the pthreads extension from PECL, because PHP does not support threading out of the box.

You can find an explaination on how to use threads with this question :

How can one use multi threading in PHP applications

Community
  • 1
  • 1
n00dl3
  • 21,213
  • 7
  • 66
  • 76
  • This isn't true. You need to use OS's even interface (`epoll` on Linux) and an extension that exposes it for your application. Threading doesn't mean asynchronous. Take a look at [Icicle](https://icicle.io), which exposes exactly what I wrote about. -1 is for incorrect answer, and there are no examples in your answer that indicate how OP can use threads to achieve his task. – Mjh Dec 19 '16 at 10:45
  • Thanks again for your response and comment. I'm going to have a look through that thread and see if this, or @Avihay m 's answer is best suited (and easiest to implement) in my case. – Ben Rees Dec 19 '16 at 10:46
  • How the hell is it incorrect ? "threading" does not mean "asynchronous", I agree,-JS is single-threaded but has asynchronous mechanism- but threading is a way to implement a "do something, notify later" pattern. @Mjh – n00dl3 Dec 19 '16 at 11:01
  • It's incorrect because someone asked you how to build a house and you answered with "use a hammer". It's incorrect because using threads doesn't mean you achieved asynchronicity. The OP also told you he's a novice in PHP but your suggestion is to use threads - what are the chances he'll do it correctly? If you're suggesting to install an extension for PHP, why not suggest an event loop, which hides implementation but exposes async operations? This is exactly what `node.js` does, and one can achieve the same thing in PHP via extensions. Take a look at the library I linked. – Mjh Dec 19 '16 at 11:08
  • Thanks for both your suggestions @Mjh and n00dl3(@) - I will go away and research what is going to be the most appropriate. I try to avoid installing libraries where possible (so i understand everything) but in this case it seems to be the only option. I'll mark the answers as soon as I can :) – Ben Rees Dec 19 '16 at 11:41