How to achieve progressive response using curl

Question

I'm building a project in PHP which the system sends a curl request to another domain for processing a query. Eg. http://domain.com/process_query?q=abc. The process_query controller executes the query against a set of files and returns the summary results of individual files in the following format.

[
    array('filename' => "one.dat", 'summary_counts' => 5),
    array('filename' => "two.dat", 'summary_counts' => 30),
    array('filename' => "three.dat", 'summary_counts' => 4),
    array('filename' => "four.dat", 'summary_counts' => 70),
    array('filename' => "five.dat", 'summary_counts' => 0)
]

It is then captures by the system which sent the curl request and renders it on the screen. I wish to do this in a progressive way. Like, as soon as the process_query controller completes processing a file, it should be able to echo the response [array('filename' => "one.dat", 'summary_counts' => 5)] and continue processing the next file. If this is possible, for every response received the system can show a progressive query processing. Can this be achieved using PHP-curl. If not what is the best alternative way?

curl would block while each request is "in flight", so wouldn't `foreach(...) { curl ...; echo done }`-type stuff do what you want? — Marc B, Dec 10 '15 at 18:42
@MarcB you want me to send a separate curl request for every file? — DhiwaTdG, Dec 10 '15 at 18:54
I am doing it by using socket.io and have done it using TCP... both are fast but if you ask me to this in PHP i would not recommend it. I say go for node.js > http://socket.io/ — Rohit Hazra, Dec 11 '15 at 14:52

score 0 · Answer 1 · answered Dec 11 '15 at 13:39

0

Yes it is possible. There is a library https://github.com/stil/curl-easy Visit this gitHub library and check the documentation on how to implement it!

Hope it helps :-)

answered Dec 11 '15 at 13:39

Rafique Mohammed

3,666
2
38
43

I don't think the original question was about async/parallel curl requests (although that might help improve performance), I think it was about sending progress back to the browser after each step was completed. – Simon East Nov 09 '17 at 00:39

score 0 · Answer 2 · answered Nov 09 '17 at 00:50

There are probably tools like the one Rafique mentioned to help improve performance by running curl requests in parallel, although if your main goal is to send a progress report after each step in the task is run, you can simply echo and then flush() the response to the browser.

Here is a super simple script that demonstrates the ability to flush content through to the browser part-way through a script:

<?php
echo "1\n";
ob_flush();
flush();

sleep(1);
echo "2\n";
ob_flush();
flush();

sleep(1);
echo "3\n";

When you access this from command line like curl test.php you'll see the numbers appearing 1, 2, 3 one second apart from each other.

You can use this technique combined with AJAX to trigger a long action from your webpage and then display some kind of progress/status update on the page. You'll probably need to hook into the onprogress event (see this thread) or you could otherwise trigger it inside an iframe and see the output display progressively (making sure you pad it as discussed here).

How to achieve progressive response using curl

2 Answers2