Best way to get data from others services

Question

I have array, for example:

$links = array(
     'http://aaa.com/data.txt',
     'http://aaea.com/data.txt',
     'http://aada.com/data.txt',
     'http://agaa.com/data.txt',
     'http://ahaa.com/data.txt',
     'http://awha.com/data.txt',
     'http://aaeha.com/data.txt',
     //etc x100
);

And in PHP I am doing:

foreach ($links as $link) {
    $data = file_get_contents($link);
    //save data in database
}

It works fine, but terribly slow. How is better way for this with PHP? I would like get data asynchronous.

My other way - jQuery and Ajax queries from PHP script, but maybe exists better way?

PHP is single-threaded (though there are some multi-thread libraries), so you won't be able to asynchronously fetch very easily within a script. other options include caching those data files and only refreshing them if they are out of date (you can fetch the header from the URL to try to find the datetime). — fbas, Apr 06 '17 at 16:55
You may find this helpful: http://stackoverflow.com/questions/15559157/understanding-php-curl-multi-exec — S. Imp, Apr 06 '17 at 17:08

Nicholas English · Answer 1 · 2017-04-06T18:11:56.360

I would suggest doing it this way.

<?php
$Links = array(
  'http://aaa.com/data.txt',
  'http://aaea.com/data.txt',
  'http://aada.com/data.txt',
  'http://agaa.com/data.txt',
  'http://ahaa.com/data.txt',
  'http://awha.com/data.txt',
  'http://aaeha.com/data.txt'
);
$TempData = '';
foreach ($Links as $Link) {
  $TempData .= file_get_contents($Link);
  $TempData .= '|';
}
$Data = rtrim($TempData, '|');

// save the $Data string and when you export the
// string from the db use this code to turn it into an array
//
// $Data = explode('|' $ExportedData);
// var_dump($Data);
//
// If you do it this way you will be preforming 1 sql
// statement instead of multiple saving site resources
// and making you code execute faster

?>

If this helped let me know.

I think a bottleneck is calling multiple file_get_contents when PHP has to wait for response from remote server and not multiple database queries. — LiTe, Apr 06 '17 at 18:35

Best way to get data from others services

1 Answers1