file_put_contents too slow when handling large file or multiple files

Question

I am using file_put_contents to create a video file. the problem is the speed and performance. It takes about an average of 30 to 60 minutes for an average file size of 50 mb to be created and that is just for one file alone. I am decoding a byte array to create the file. How can I improve the speed and performance?

$json_str = file_get_contents('php://input');
$json_obj = json_decode($json_str);
$Video = $json_obj->Video;
$CAF = $json_obj->CAF;
$Date = $json_obj->Date;
$CafDate = date("Y-m-d", strtotime($Date));

$video_decode = base64_decode($Video);
$video_filename = __DIR__ . '/uploads/'. $CAF . '_'.$CafDate.'_VID.mp4';
$video_dbfilename = './uploads/'. $CAF . '_'.$CafDate.'_VID.mp4';
$save_video = file_put_contents($video_filename, $video_decode);

Answer 1

You should not load entire files to the memory when you can't foresee the size or when it's going to handle huge files. It's better to read the file in chunks and process it chunk by chunk.

Here's a quick and dirty example of how to achieve it:

<?php
// open the handle in binary-read mode
$handle = fopen("php://input", "r");

// open handle for saving the file
$local_file = fopen("path/to/file", "w");

// create a variable to store the chunks
$chunk = '';

// loop until the end of the file
while (!feof($handle)) {
  // get a chunk
  $chunk = fread($handle, 8192);

  // here you do whatever you want with $chunk
  // (i.e. save it appending to some file)
  fwrite($local_file, $chunk);
}

// close handles
fclose($handle);
fclose($local_file);

Answer 2

There are many problems here, with a compounding effect.

$json_str = file_get_contents('php://input');

This reads the entire input into memory, and blocks until it's done. Don't do this.

$json_obj = json_decode($json_str);

If you have 50MB of JSON, you're probably doing something wrong.

$Video = $json_obj->Video;

This is what you're doing wrong... storing a large binary blob in a small structured data serialization format. You realize that the JSON parser has to read and handle the entire thing? Don't send JSON for this!

$video_decode = base64_decode($Video);

Don't use base-64 encoding unless absolutely necessary. It adds 33% overhead to the storage size, as well as CPU for encoding/decoding. This is a complete waste, and is totally unnecessary.

$video_filename = __DIR__ . '/uploads/'. $CAF . '_'.$CafDate.'_VID.mp4';
$video_dbfilename = './uploads/'. $CAF . '_'.$CafDate.'_VID.mp4';

You may have a serious security issue with those two lines. What if someone posts a path of ../../../etc/init.d/something-evil? Don't let the user dictate the filename on disk in any way.

$save_video = file_put_contents($video_filename, $video_decode);

The comment from file_get_contents() applies here in that you should be using this method. Stream the contents to disk instead.