0

I want to get about 50 variables from several different sites and show them to user, with PHP Simple HTML DOM Parser. My Code Structure is something like this:

                    <?php


                    include('simple_html_dom.php');



                    $html = file_get_html('https://site1.com/name1/');

                        foreach ($html->find('span#name1') as $e) {
                            $name1 = $e->outertext;
                            echo $name1;
                            break;
                        }

                    $html = file_get_html('https://site2.com/family1/');

                        foreach ($html->find('span#family1') as $e) {
                            $family1 = $e->outertext;
                            echo $family1;
                            break;
                        }

                    $html = file_get_html('https://site3.com/degree1/');

                        foreach ($html->find('span#degree1') as $e) {
                            $height1 = $e->outertext;
                            echo $height1;
                            break;
                        }

                    $html = file_get_html('https://site4.com/height1/');

                        foreach ($html->find('span#height1') as $e) {
                            $height1 = $e->outertext;
                            echo $height1;
                            break;
                        }


                    // About 30 other blocks of code similar to the above code

                    ?>

At the moment, this code works well and displays the variables, but as expected, it takes about two or three minutes to do this process and show all the variables to the user, which is a lot of time.

Now I'm looking for a way to show to the user (send result to browser), each variable that was found and then find the next variable.

For example, when the value of $name1 is specified, show it to the user and then go to specify the value of $family1

Apparently I should use flush() to fix this problem, but unfortunately I could not find a solution to fix this problem.

h1h2
  • 85
  • 2
  • 11
  • 1
    Does this answer your question? [How to flush output after each \`echo\` call?](https://stackoverflow.com/questions/3133209/how-to-flush-output-after-each-echo-call) – Nico Haase Sep 28 '20 at 07:31
  • Could look at `curl_multi` or some other async approach. But overall I think you are probably on the right track breaking it up into steps. Could just do 10 per page and have 5 steps. Make it seem faster. If these things are more static you could load them before hand and cache too. – ficuscr Sep 28 '20 at 07:32
  • @NicoHaase Thanks for your guidance But unfortunately I had seen this question before but I could not solve my problem with it – h1h2 Sep 28 '20 at 07:34

2 Answers2

1

You will most likely need to architect your app slightly differently. A relatively simple approach would be to to an AJAX call for each variable you need to show, and let the client (browser) decide when to make each call so you can parallelize them. Each call would hit one (and only one) remote URL. This way, you can display results as they come in as AJAX responses.

A slightly more complex approach would be to use Websockets to send back new information as it becomes available. But I think this might be overkill.

In any case both solutions require more browser / JavaScript work than PHP.

shevron
  • 3,463
  • 2
  • 23
  • 35
1

You can use ob_flush approach like:

<?php

include('simple_html_dom.php');

$html = file_get_html('https://site1.com/name1/');

    foreach ($html->find('span#name1') as $e) {
        $name1 = $e->outertext;
        echo $name1;
        break;
    }

    flush();
    ob_flush();

$html = file_get_html('https://site2.com/family1/');

    foreach ($html->find('span#family1') as $e) {
        $family1 = $e->outertext;
        echo $family1;
        break;
    }

    flush();
    ob_flush();

$html = file_get_html('https://site3.com/degree1/');

    foreach ($html->find('span#degree1') as $e) {
        $height1 = $e->outertext;
        echo $height1;
        break;
    }

    flush();
    ob_flush();

$html = file_get_html('https://site4.com/height1/');

    foreach ($html->find('span#height1') as $e) {
        $height1 = $e->outertext;
        echo $height1;
        break;
    }

    flush();
    ob_flush();
// About 30 other blocks of code similar to the above code

?>

This code after each block flush PHP output to browser.

Slava Rozhnev
  • 9,510
  • 6
  • 23
  • 39