Concatenate files in PHP

Question

I'd like to know if there is a faster way of concatenating 2 text files in PHP, than the usual way of opening txt1 in a+, reading txt2 line by line and copying each line to txt1.

Answer 1

If you want to use a pure-PHP solution, you could use file_get_contents to read the whole file in a string and then write that out (no error checking, just to show how you could do it):

$fp1 = fopen("txt1", 'a+');
$file2 = file_get_contents("txt2");
fwrite($fp1, $file2);

Answer 2

It's probably much faster to use the cat program in linux if you have command line permissions for PHP

system('cat txt1 txt2 > txt3');

Answer 3

$content = file_get_contents("file1");
file_put_contents("file2", $content, FILE_APPEND);

Answer 4

I have found using *nix cat to be the most effective here, but if for whatever reason you don't have access to it, and you are concatenating large files, then you can use this line by line function. (Error handling stripped for simplicity).

function catFiles($arrayOfFiles, $outputPath) {

    $dest = fopen($outputPath,"a");

    foreach ($arrayOfFiles as $f) {

        $FH = fopen($f,"r");

        $line = fgets($FH);

        while ($line !== false) {

            fputs($dest,$line);

            $line = fgets($FH);

        }

        fclose($FH);

    }

    fclose($dest);

}

Answer 5

While the fastest way is undobtedly to use OS commands, like cp or cat, this is hardly advisable for compatibility.

The fastest "PHP only" way is using file_get_contents, that reads the whole source file, in one shot but it also has some drawbacks. It will require a lot of memory for large files and for this reason it may fail depending on the memory assigned to PHP.

A universal clean and fast solution is to use fread and fwrite with a large buffer.

If the file is smaller than the buffer, all reading will happen in one burst, so speed is optimal, otherwise reading happens at big chunks (the size of the buffer) so the overhead is minimal and speed is quite good.

Reading line by line with fgets instead, has to test for every charachter, one by one, if it's a newline or line feed. Also, reading line by line with fgets a file with many short lines will be slower as you will read many little pieces, of different sizes, depending of where newlines are positioned.

fread is faster as it only checks for EOF (which is easy) and reads files using a fixed size chunk you decide, so it can be made optimal for your OS or disk or kind of files (say you have many files <12k you can set the buffer size to 16k so they are all read in one shot).

// Code is untested written on mobile phone inside Stack Overflow, comes from various examples online you can also check.

<?php

$BUFFER_SIZE=1*1024*1024; // 1MB, bigger is faster.. depending on file sizes and count

$dest = fopen($fileToAppendTo "a+");
if (FALSE === $dest) die("Failed to open destination");

$handle = fopen("source.txt", "rb");
if (FALSE === $handle) {
    fclose($dest);
    die("Failed to open source");
}

$contents = '';
while( !feof($handle) ) {
    fwrite($dest, fread($handle, $BUFFER_SIZE) );
}

fclose($handle);
fclose($dest);

?>