Echoing large amount of HTML datas takes much time PHP

Question

I have 15k of JSON products that i need to process with PHP, if i var_dump the array i have no problems. if i reduce the JSON datas to 1000-1500 it takes 1/2 seconds, and the user interface composed by two buttons that do some jquery actions takes some seconds. The problem is obviusly when i pass all datas(15k), it takes 40/50 seconds to display HTML page and if i click buttons they don't work.

this is what i do: getting JSON from an external URL, (no problems). passing JSON using AJAX(POST) to PHP, (no problems). getting string in PHP file, encode the string in array and process each data, after data_processing is okay i'm echoing datas as

<? 
   echo'<div class="myclass" id="myid">';
   echo'<input type="hidden" value="somedatas">';
   echo'<input type="text" name="'.$json[$_POST["SOMEDATA"]].'" value="somedatas">";
   ....
?>

is there a better method to do that? i also tried to set display:none to main div, but no changes.

UPDATE i forgot a pass, the php echo is turned back to an ajax request and appended to a div

this is the script that i use to process data one by one once confirm button is clicked

        echo'<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
<script>
$(document).ready(function(){
    $("#close_tab_prw").click(function(){
         document.getElementById("prd_prw_tab").innerHTML = ""; //reset div
         $("#prd_prw").css("display","none");
       $("#container_others").css("opacity","1");
}); 
     $("#update_all_items").click(function(){
        var prd_number = $("input[name=prd_max_number]").val();
        for(var i = 0; i < prd_number; i++){
            var string_ok = "#confirm_form_prd_";
            string_ok = string_ok.concat(i);
            var dataString = $(string_ok).serialize();
             $.ajax( {
        type: "POST",
        url: "';echo $dir; echo'",
        data: dataString,
        success: function(data) {
            console.log(data);
            $("#prd_prw_tab").html(data);
        }
            });
        }
    });
     $("#disable_img").click(function(){
        $(".img_check").prop("checked", false);
    });
     $("#allow_img").click(function(){
        $(".img_check").prop("checked", true);
    });
     $("#disable_desc").click(function(){
        $(".n_desc_chk").prop("checked", false);
    });
     $("#allow_desc").click(function(){
        $(".n_desc_chk").prop("checked", true);
    });


    });

</script>';

and this is the main function that i use to display products:

$(document).ready(function(){
 $("#get_json_btn").click(function(){


        document.getElementById("unique_json_return").innerHTML = ""; //reset div

        if(document.getElementById('url_json').value != "") {
            $.getJSON(document.getElementById('url_json').value , function(result){
            $.each(result, function(i, field){
                $("#unique_json_return").append(JSON.stringify(result));
            })
        })
        .fail(function() { alert('getJSON request failed, see tips '); })
        }
       else {
            alert("No Url insrted, please, insert your JSON url");
        } 
    });
 $("#update_products").click(function(){
      new_height = "";
    $( window ).resize(function() {
        $("#prd_prw").css("height", $(window).height()-(20*$(document).height()/100));
    });
      new_height = $(window).height()-(20*$(document).height()/100);
      $("#prd_prw").css("height", new_height);
     $("#prd_prw").css("display","block");
       $("#container_others").css("opacity","0.3");

    var entire_json = document.getElementById('unique_json_return').innerText;
    var one = document.getElementsByName('one')[0].value;
    var two = document.getElementsByName('two')[0].value;
    ...
    var ten = document.getElementsByName('ten')[0].value;


    if(entire_json == "") {
        alert("Invalid or Empty JSON response");
    }
    else{
                $.ajax({
   url: '<?echo $dir; ?>',
   data: {entire_json : entire_json, one:one, two:two, three:three,...ten:ten
    },
   type: 'POST',
   success: function(data) {
   document.getElementById("unique_json_return").innerHTML = ""; //reset div
    $("#prd_prw_tab").append(data);

       }
});
    }
});
});

you dont need to explicitly echo html to the page. [this question has some good information](http://stackoverflow.com/questions/4148031/what-is-the-best-practice-to-use-when-using-php-and-html) — castis, Feb 07 '17 at 14:50
your echos are not useful. Add the code that you are explaining. We understand better the code than your words. Here can be a javascript parser problem (with a low performance PC) or can be a bad coding of the PHP. We can't know. — Marcos Pérez Gude, Feb 07 '17 at 14:50
Without entirely understanding your problem, consider paginating your result set. See http://stackoverflow.com/questions/3705318/simple-php-pagination-script — sellmeadog, Feb 07 '17 at 15:22
@AndreaLoda stackoverflow's rules said that you need to attach a minimal, complete and verifiable example, but not the whole code. See more: [**How to ask**](http://stackoverflow.com/help/how-to-ask) || [**MCVE**](http://stackoverflow.com/help/mcve) — Marcos Pérez Gude, Feb 07 '17 at 15:24
when i turn my data back from php to ajax succes function with only 3k items, the amount of datas in console is 10mb — Andrea Loda, Feb 07 '17 at 15:30

score 2 · Answer 1 · answered Feb 07 '17 at 14:51

2

How often are you required to parse these 15k products?

If it's just the occasional execution, on the back-end only and no end users experience the 50 second wait, I would just tolerate it until you can split it out into a separate service (possibly not PHP).

If this is something that happens after end-user interaction, I would revisit the entire design and ensure that no single user action results in parsing 15k objects in their entirety.

answered Feb 07 '17 at 14:51

HomerPlata

1,687
5
22
39

how? can you make an example? – Andrea Loda Feb 07 '17 at 15:07
I'd be happy to have a go, if you could tell me more detail about what you're trying to achieve. What are these 15k objects, why do they all need processing at once, how often is this required, and what is the end goal of this task? – HomerPlata Feb 07 '17 at 15:29
i'm sending 15k objects that contains some objects details, i'm getting them JSON formatted from an URL using AJAX, then i pass them clicking a button with another AJAX request as updated above to a php file, that process them, analyizing each data one by one, and printing them, i need to process them at once because when data (at the end of process) are passed back to the success function, the ajax prints them, user can modify them using and then with a button process another ajax request to another PHP function that process each item one by one and store them. – Andrea Loda Feb 07 '17 at 15:38
Hmmmm, why do you need to process 15k at once EVERY time? Can you not dump them into a database ONCE and then run queries against that? If that's not possible because the 15k record response from that URL is subject to change, is there not a more specific way of querying data from that source? It seems like a flaw in design to require pulling all the data in every time, and then process sub sections of it. Only grab what you need. – HomerPlata Feb 07 '17 at 15:43
The whole problem is the execution, but your specific problem I think is javascript. I said it on the main comments. – Marcos Pérez Gude Feb 07 '17 at 15:44
just check @MarcosPérezGude – Andrea Loda Feb 07 '17 at 15:54
DOM tree is exhausted with your code. You are empty and filling all times 10 MB of data to the DOM. The weird thing is that this doesn't crash your browser. The DOM needs to read your html string (yeah, are not elements yet), and then parse them to convert to DOM objects. Adding the PHP time on server process to the Javascript time to client process, this task can turn crazy the user that manages it. – Marcos Pérez Gude Feb 07 '17 at 16:08
The best thing you can do is what others are telling you: dump the json to a database and make queries against it. Add a pagination, or something, just think how to optimize the process on client and the code will appears magically on your head. – Marcos Pérez Gude Feb 07 '17 at 16:09
and what if i try to send datas one by one to php? by turn string entire_json to a real json obj and attaching datas to div one by one? @MarcosPérezGude – Andrea Loda Feb 07 '17 at 16:18
You can't make it from php. You need to make one ajax request to php per item. I think that's poor that your current code. Will be slower. – Marcos Pérez Gude Feb 07 '17 at 16:33

Echoing large amount of HTML datas takes much time PHP

1 Answers1