reencode non-fragmented mp4 into fragmented mp4

Question

I'm using MediaSource extension to play videos. However,MediaSource only supports fragmented MP4 videos.

How can I convert a non-fragmented MP4 video into a fragmented MP4 video in JavaScript so that I can use MediaSource to play the video?

MP4Box's isFragmented boolean returns false on non-fragmented videos:

blob.arrayBuffer().then((videoBuffer => {
    mp4boxfile.onReady = function (info) {
        const isFragmented = info.isFragmented;
        if (!isFragmented) {
            //Need to reencode video into a fragmented video here
        }
    }

//...

    const chunkSize = Math.ceil(videoBuffer.byteLength / 80000);
    let start = 0;
    let currentEnd = 80000;
    let videoBufferChunk;

    for (let i = 0; i < chunkSize; i++) {
        videoBufferChunk = videoBuffer.slice(start, currentEnd);

        videoBufferChunk.fileStart = start;
        start = mp4boxfile.appendBuffer(videoBufferChunk);
        currentEnd = start + 80000;
        mp4boxfile.flush();
    }
//...
}

How can I reencode a MP4 video into a fragmented one with JavaScript?

Shouldn't you transcode the video on the server and then use the fragmented version? One of the answers that you cite leads to instructions about [how to fragment an MP4 with ffmpeg](https://developer.mozilla.org/en-US/docs/Web/API/Media_Source_Extensions_API/Transcoding_assets_for_MSE#Fragmenting). — Hernán Alarcón, Jan 02 '21 at 01:22
@HernánAlarcón I could, but that won't work. This application as it is designed to be front-end only so it must be done on the front-end. — now_world, Jan 02 '21 at 01:29
@orangeMint The _"How?"_ explanation of MP4 structure and example code for handling its data is too long for an S.O post. **(1)** Why do you need to play the MP4 file via MSE? There might be another way to achieve same expected end-result... **(2)** Does MP4Box-JS do segmenting? If yes, that'll be easier than asking about writing your own code)... **(3)** If you want to try write the code yourself, tell us what you need to understand. — VC.One, Jan 02 '21 at 02:37
@VC.One 1:There may be another way to do this I'm not sure. I've tried a few other ways, but this seems to be the best option since it does already work with all fragmented mp4 files that I have. 2:I'm currently only using MP4Box to check if the video is already fragmented and to get the videos codec mime information so that I can store it for later use. 3:I'd prefer to use a library that will do the reencoding for me, but I haven't found one that does it (at least yet). Im happy to explain more,but this is just a small part of the larger picture and to put the whole thing on SO is a bit much. — now_world, Jan 02 '21 at 02:50
@orangeMint What I'm asking (at point 1) is this: Most coders use MSE to switch stream qualities or do a livestream etc.. You have one mp4 file, why does it need MSE and not just a video tag? What problem are you trying to fix with using MSE? Things like stepping frame-by-frame, changing audio stream (of a video), or loading video bytes from Array (not URL) can all be done without MSE... — VC.One, Jan 02 '21 at 15:24
PS: I saw your [other question](https://stackoverflow.com/q/65334051/2057709). Regarding about your comment _"some videos play just fine. Others, give a warning"_. I suspect the error was caused by the video file not providing metadata header before giving video data. Some MP4 files have their metadata stored at the back end of file so MSE decoder not getting bytes "moov" (video meta) before it finds bytes "mdat" (media data) will error.. It's possible the solution to playing your MP4 (without fragmenting) in MSE is to just move the header to the front of MP4 (using a tool). — VC.One, Jan 02 '21 at 16:25
@VC.One Interesting. How could I go about testing this? What tool are you referring to? — now_world, Jan 03 '21 at 21:37
@orangeMint I will test it for you and report back as soon as possible (within 24 hrs, if not sooner). — VC.One, Jan 11 '21 at 16:20
@orangeMint Did I understand you right you need something like streaming video by its byte array partitions? — Daniil Loban, Jan 14 '21 at 11:49
https://gpac.github.io/mp4box.js/#segmentation does it not suit you? — Daniil Loban, Jan 14 '21 at 11:59

score 2 · Accepted Answer · answered Jan 14 '21 at 22:42

You could use ffmpeg.wasm, it's basically ffmpeg for the browser but with native performance. This way you could integrate the fragmentation into your front end which is what you are trying to do, if I understand correctly.

ffmpeg.wasm

I am afraid I can't help you with the actual fragmentation part using ffmpeg, but this post discusses it for normal ffmpeg.

reencode non-fragmented mp4 into fragmented mp4

1 Answers1