0

I am trying to speed-up the file creation of my video. My problem is file_put_contents takes to long for an average of 50mb video file to be created. My code below will first decode the video and put it inside the directory. $Video is a byte array. Is there a way to create the video file without base64 decoding and what can I do to speed the file creation?

$json_str = file_get_contents('php://input');
$json_obj = json_decode($json_str);

$CAFNo = $json_obj->CAFNo;
$Date = $json_obj->CAFDate;
$Video = $json_obj->Video;
$CafDate = date("Y-m-d", strtotime($Date));

$video_decode = base64_decode($Video);
$video_filename = __DIR__ . '/uploads/'. $CAFNo . '_'.$CafDate.'_VID.mp4';
$save_video = file_put_contents($video_filename, $video_decode);

$video_dbfilename = './uploads/'. $CAFNo . '_'.$CafDate.'_VID.mp4';

$sql = "UPDATE tblCaf SET Video = '$video_dbfilename' WHERE CAFNo = '$CAFNo'";
mysqli_query ($conn, $sql);
  • Define "takes too long". How big is the file that you are creating? – Dave Dec 20 '18 at 14:53
  • average file size of 50mb @Dave –  Dec 20 '18 at 14:53
  • Speed is going to be related to the hardware you are running the code on. Too long is a very subjective thing regardless. – Dave Dec 20 '18 at 14:54
  • @Dave is there something I can do in my code to atleast speed things up just a little? –  Dec 20 '18 at 14:55
  • @JonStirling can I use fwrite() instead? is it faster? –  Dec 20 '18 at 14:58
  • Both `file_get_contents()` and `file_put_contents()` imply loading the complete video in RAM. There's no gain in using such functions, good old `fopen()` and `fread()` are the only sensible mechanism. (Why Base64 at all?). – Álvaro González Dec 20 '18 at 14:59
  • 1) Probably not 2) Why not try it and see rather than asking me 3) I'm not convinced you read my comment. – Jonnix Dec 20 '18 at 14:59
  • @ÁlvaroGonzález how can i use it can you show me just to get the general idea –  Dec 20 '18 at 15:01

1 Answers1

1

A media file can be huge so you cannot process it with techniques that involve loading the whole file in RAM, let alone have several copies of the data at the same time. Your code is doing that several times.

$json_str = file_get_contents('php://input');
$json_obj = json_decode($json_str);
$Video = $json_obj->Video;
$video_decode = base64_decode($Video);

At this point, you have 4 copies of the video loaded in RAM. For a 50 MB media file that means that $video_decode holds 50 MB of data and the other variables are even larger. You also need to add up the internal memory used by functions like json_decode(). To get an idea of the RAM being used you can insert this between each operation:

var_dump(memory_get_usage(true));

Additionally, modern video codecs are highly optimised binary streams. If you convert that binary data to plain text the size will grow. Base64 makes data grow by 4/3. JSON adds a little overhead, though nothing serious at this point.


If I had to design this from scratch I'd use a POST request with a binary format (either multipart/form-data or a custom encoding, maybe just the file as-is) and then just read input in chunks:

If you prefer or need to stick to current design, you need to address all the file reads and decoding operation in chunks:

Last but not least, don't ever create SQL code by injecting untrusted external input:

$sql = "UPDATE tblCaf SET Video = '$video_dbfilename' WHERE CAFNo = '$CAFNo'";

Use prepared statements instead. The hordes of tutorials that suggest otherwise should have been burnt long ago.


Update: It seems that the concept of processing data in small chunks (vs. loading everything in memory) versus is not as straightforward as I thought so I'll provide runnable code to illustrate the difference.

First of all, let's produce a 100 MB file to test:

$test = __DIR__ . '/test.txt';
$fp = fopen($test, 'wb') or die(1);
for ($i = 0; $i < 1839607; $i++) {
    fwrite($fp, "Lorem ipsum dolor sit amet, consectetur adipiscing elit.\n") or die(2);
}
fclose($fp);

If you resort to easy-to-use one-liner helper functions:

ini_set('memory_limit', '300M');
$test = __DIR__ . '/test.txt';
printf("Current memory usage: %d MB\n", memory_get_usage(true) / 1024 / 1024);
$data = file_get_contents($test);
printf("Current memory usage: %d MB\n", memory_get_usage(true) / 1024 / 1024);
file_put_contents(__DIR__ . '/copy.txt', $data);
printf("Current memory usage: %d MB\n", memory_get_usage(true) / 1024 / 1024);
printf("Maximum memory usage: %d MB\n", memory_get_peak_usage(true) / 1024 / 1024);

... it comes at the price of requiring a potentially large amount of memory (there's no thing as free food):

Current memory usage: 0 MB
Current memory usage: 100 MB
Current memory usage: 100 MB
Maximum memory usage: 201 MB

If you split the file in small pieces:

$test = __DIR__ . '/test.txt';
$chunk_size = 1024 * 1024;
$input = fopen($test, 'rb') or die(1);
$output = fopen(__DIR__ . '/copy.txt', 'wb') or die(2);
printf("Current memory usage: %d MB\n", memory_get_usage(true) / 1024 / 1024);
while (!feof($input)) {
    $chunk = fread($input, $chunk_size) or die(3);
    fwrite($output, $chunk) or die(4);
}
printf("Current memory usage: %d MB\n", memory_get_usage(true) / 1024 / 1024);
fclose($input);
fclose($output);
printf("Current memory usage: %d MB\n", memory_get_usage(true) / 1024 / 1024);
printf("Maximum memory usage: %d MB\n", memory_get_peak_usage(true) / 1024 / 1024);

... you can use a fixed amount of memory for files any size:

Current memory usage: 0 MB
Current memory usage: 1 MB
Current memory usage: 1 MB
Maximum memory usage: 3 MB
Álvaro González
  • 142,137
  • 41
  • 261
  • 360