In an application that consists in different MediaCodec and (others Android.Media namespace stuff) prooves of concept, I'm generating a video on an opengl frame along with a PCM tone so that I have data to send to a two MediaCodec encoder (PCM chunks sent to a MediaCodec audio encoder and a frame with surfacecolorformat sent to a MediaCodec video encoder).
You see me coming: I build AVC/mp4a movies. These generated movie play nice in any player.
I'm using MediaMuxer to merge videos together. I know that the MediaFormat of the videos need to be identical to be able to merge them together. However, let's say I have these movies:
A Happy_cat.mp4 320x20024fps1 i-frame per secondSource: Interwebz
B Yellow background.mp4320x20024fps1 i-frame per secondSource: Me
With MediaMuxer, I can mix any splitted or full version of A together, or B together, but I can't merge A with B.
If I concatenate A(1) + B(1) + A(2) + B(2), I'll see players playing A(1) then skip to A(2), skipping the exact duration of B. In other words, it will show the cat, then skip to the cat without showing the yellow background. Same if I concatenate B(1)+A(1)+B(2)+A(2). I'll see yellow with my sound, it will skip the cat, then will show the yellow, then stop before playing the cat.
Knowing that movies need to share the same format, I have a reference movie from which I "copy" MediaFormat parameters using MediaExtractor.
Am I limited to movies that have the same origin because of parameters I cannot set in MediaFormat before adding the track to the muxer? Are csd-0 and csd-1 importants in MediaFormat? Because I think it may differ between videos. That's what I worry the most when comparing videos. I fear MediaMuxer could digest these data in csd-0 in some native API methods, as well as bug if there is media config differences based on data in csd-0/csd-1. [Update] I'm more and more convinced this is a problem with csd after reading http://bigflake.com/mediacodec/ section about csd-x. The only solution would be to transcode everything. :(
Data from a B(1) A(1) B(2) A(2) Merge
B(1) Yellow:
format {{max-input-size=266, durationUs=2809569, channel-count=1,
mime=audio/mp4a-latm, csd-
0=java.nio.ByteArrayBuffer[position=0,limit=2,capacity=2], sample-
rate=44100}}
format {{max-input-size=431, durationUs=3041666, csd-
1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], height=480,
mime=video/avc, csd-
0=java.nio.ByteArrayBuffer[position=0,limit=18,capacity=18], width=270}}
A(1) Cat:
format {{max-input-size=23243, durationUs=7000000, csd-
1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], height=480,
mime=video/avc, csd-
0=java.nio.ByteArrayBuffer[position=0,limit=14,capacity=14], width=270}}
format {{max-input-size=220, durationUs=7066621, channel-count=1,
mime=audio/mp4a-latm, csd-
0=java.nio.ByteArrayBuffer[position=0,limit=2,capacity=2], sample-
rate=44100}}
B(2) Yellow:
format {{max-input-size=266, durationUs=2809569, channel-count=1,
mime=audio/mp4a-latm, csd-
0=java.nio.ByteArrayBuffer[position=0,limit=2,capacity=2], sample-
rate=44100}}
format {{max-input-size=431, durationUs=3041666, csd-
1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], height=480,
mime=video/avc, csd-
0=java.nio.ByteArrayBuffer[position=0,limit=18,capacity=18], width=270}}
A(2) Cat:
format {{max-input-size=23243, durationUs=7000000, csd-
1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], height=480,
mime=video/avc, csd-
0=java.nio.ByteArrayBuffer[position=0,limit=14,capacity=14], width=270}}
format {{max-input-size=220, durationUs=7066621, channel-count=1,
mime=audio/mp4a-latm, csd-
0=java.nio.ByteArrayBuffer[position=0,limit=2,capacity=2], sample-
rate=44100}}
[Update]
In my original source code, I was not setting the tracks order properly. After having fixed that, I can say that anything produced with the phone (MediaCodec/MediaMuxer/3rd party app) will have the same audio and video CSDs and will merge gracefully. Any external AVC movie will generally have different CSDs and if so, will just not merge, thus making transcoding from that supported/playable AVC to the phone CODEC AVC mandatory before being able to Merge. Note that anything that can be played can be used with all other MediaCodec features. It just makes it less trivial to play around with that video since it was encoded with an unknown encoder. There may be some possibilities to duplicate the CSDs from the movie coming from an external source, but I haven't been successful.
To dermine if the movie was encoded with the owner phone or another phone, when launching my app, I create an empty movie and save the captured CSDs to a file. Then I can compare it with any movie. It would be interesting to have a well developed helper that can humanize the CSD data so that at least it is easily understood and easier to adapt an encoding to that source.
