How can you stream results from an API in PHP

Question

is there a way in PHP, perhaps with an external library, to stream results from an API that responds with JSON data?

For instance I have the following code to get the data:

$resultsAPI = "https://www.example.com/api/results.json? 
app_id=$app_id&token=$token&page=1&limit=10";
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $resultsAPI);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Accept: application/json;api_version=2' ));
$resp = curl_exec($curl);
curl_close($curl);
$results = json_decode($resp, true)['results'];

foreach ($results as $key=>$resultImage) {
    $resultImage= "$resultImage[images]?app_id=$app_id&token=$token";
    $curl = curl_init();
    curl_setopt($curl, CURLOPT_URL, $resultImage);
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
    $resp = curl_exec($curl);
    curl_close($curl);
    $image = json_decode($resp, true);
    $results[$key]['image1'] = $image['image1'];
}

echo '<div class="card"><ul>';
foreach ($results as $result) {
echo '<li>';
echo '<span><p>'.$result['title'].'</p></span>';
echo '<span><p>'.$result['description'].'</p></span>';
echo '<span><img src="'.$result['image1'].'"></span>';
echo '</li>';
}
echo '</ul></div>';

It can take some time to load all data because it is going to loop over some large files. Is it possible to start streaming the results when it has the first data?

In the image below it shows what I am trying to explain. The data is being loaded in to the skeleton one by one:

Any thoughts on this would be very helpful and or if it is possible at all.

Answer 1

I think you're run into the wrong direction.

HTML begin to render after load all the html. So fetch the html with stream is not work for you.

In the demo, it just load a simple html page. Then load the other parts of the page with something like ajax. Each time a part loaded then render it.

Answer 2

Why not merge the the foreach loops? I haven't tested this but items should be echoed very iteration.

echo '<div class="card"><ul>';
foreach ($results as $key=>$result) {
    $resultImage= "$result[images]?app_id=$app_id&token=$token";
    $curl = curl_init();
    curl_setopt($curl, CURLOPT_URL, $resultImage);
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
    $resp = curl_exec($curl);
    curl_close($curl);
    $image = json_decode($resp, true);
    $results[$key]['image1'] = $image['image1'];

   echo '<li>';
   echo '<span><p>'.$result['title'].'</p></span>';
   echo '<span><p>'.$result['description'].'</p></span>';

   if(isset($image['image1'])){
       echo '<span><img src="'.$result['image1'].'"></span>';
   }
   echo '</li>';
}
echo '</ul></div>';

I hope that helps

Answer 3

As Kris Roofe said you may need ajax or axios call to output your data with some effects and animations exactly like the image that you attached in your question, your application need to be rendered first with all of it's HTML tags and assets like CSS and Javascript files then you can output some data and show it with animations by using ajax or axios and ofcurse you have more control over your data streaming in this case by using ajax or axios in client-side but you can also do it in your server-side but typically I prefer to do these things in client-side.

by the way if you insist to doing this in this way you can use flush() and ob_flush() to immediately output your data before the while loop ends. Someone has already mentioned this in php official documentation link. You can check and read full documentation about output buffering and these methods.

You should merge your two foreach loops together and then add these two methods add the end of your loop so it will make it to output your data immediately after each loop iterate.

foreach($results as $key=>$resultImage){
    //fetch images data such as title, description, and image itself in this loop
    // and aslo echo your html tags in here. 
    // echo '<li>';
    // echo '<span><p>'.$result['title'].'</p></span>';
    // echo '<span><p>'.$result['description'].'</p></span>';
    // echo '<span><img src="'.$result['image1'].'"></span>';
    // echo '</li>';

}

I wrote some comments in your for loop to show you that you should merge your loops together, because you are trying to initialize $results variable in your first loop, and then after finishing that loop you are iterating in $results variable to show output data. so you can't output data immediately with two loops in here because your second loop depends on first one and it will not start iterating until the first one finishes. check this little code that I wrote to demonstrate the usage of these two methods:

$curl = curl_init();
for($i=0;$i<5;$i++){
    curl_setopt($curl, CURLOPT_URL, 'example.com');
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
    $output = curl_exec($curl);
    $output = json_decode($output);
    foreach($output as $key=>$value){
        echo 'key: '.$key;
        echo '$value: '.$value;
        ob_flush();
        flush();
    }
}

I would highly recommend you to read about output buffering to implement it correctly in your projects.

I hope this could help you.

Answer 4

HTML starts rendering when some of it arrives. It does not have to be complete source. In PHP you can "send what you already echoed" via ob_flush() and flush() calls. This way it will immediately display in the browser. This can be paired with JSON stream parsing using for example halaxa/json-machine, so the result can look something like this:

<?php
echo '<div class="card"><ul>';
foreach (JsonMachine::fromStream($jsonStreamResource) as $result) {
    echo '<li>';
    echo '<span><p>'.$result['title'].'</p></span>';
    echo '<span><p>'.$result['description'].'</p></span>';
    echo '<span><img src="'.$result['image1'].'"></span>';
    echo '</li>';
    ob_flush();
    flush();
}
echo '</ul></div>';

Answer 5

Fetch limited records while rendering html first load. Once page load fully then call a ajax function which will fetch next page rows. Definitely it is the tested method.

    <script>
    var items = [{item1}, {item2}];
    $(document).ready(function() {
    $.each(items, function(index, item) {
    $('.card').append('<li>'+ '<span><p>'+item['title']+'</p></span>' + 
    '</li>');
     });
     });`enter code here`
    </script>

OR You may call ajax function for first page records after html rendered fully.

How do I load a JSON object from a file with ajax?