2

I need to decode data from interleave stream (like AVI).

So, it is means that I have a container that build in such shape(sequence) 

mp4 --> other data --> mp4 --> other data --> so on...

For this moment I have a basic implementation of MediaCodec decoder. I have MediaExtractor where I call setDataSource() and it get mp4 file uri as param. Then in order to get a decode result I need queue/dequeue buffer.

So, it means that byte producer is MediaExtractor which know how to extract bytes from uri.

But in my purpose I have a file(container), that consist a few types of data in sequence...

What I want to do - I will read just a mp4 chunk(one video frame) and then I need to pass this chunk(frame) to MediaCodec to process with decoding. I want to be a supplier myself.

So, question is how to pass byte buffer to MediaCodec in order to process with decoding?

P.S. There is still no decision if our interleave container will have each mp4 chunk as separate mp4 file (including headers), or just first frame in sequence...

Or maybe for this reason I need to use ffmpeg lib?

I hope I didn't miss noting anyway feel free to ask

EDIT

Meanwhile I found such solution (not final)

AMediaExtractor *ex = AMediaExtractor_new();

FILE *fp = fopen(filename.c_str(), "rb");
fseek(fp, 0, SEEK_END);
long size = ftell(fp);
fseek(fp, 0, SEEK_SET);
auto err = AMediaExtractor_setDataSourceFd(ex, fileno(fp), 0, size);

setDataSourceFd() this method knows how to work with file descriptor. He get such params as extractor, descriptor, offset and size of data.

So in my case as I told I have such data sequence 

mp4 --> other data --> mp4 --> other data --> so on...

Let's first mp4 data starts at the begging, so it is means no offset, so setDataSourceFd(ex, fileno(fp), 0, dataSize), let's say second mp4 data starts after 5000 bytes... sosetDataSourceFd(ex, fileno(fp), 5000, dataSize)

It is means that if you have for exapmle 3000 mp4 parts everytime you need to create new MediaExtractor and setDataSourceFd()...

I am not sure if it is a right way to make it...

Sirop4ik
  • 4,543
  • 2
  • 54
  • 121
  • I'm unclear. It sounds like you're feeding MediaCodec from MediaExtractor, which is pretty common. What are you doing that's different from typical .MP4 playback? – fadden Sep 16 '19 at 14:49
  • @fadden I meant that typicaly if you want playback mp4 file you need to create extactor object pass it to codec and extractor will extract frames one by one for you, but in my case I will read bytes from file for myself and somehow I need to pass this bytes to codec to decode them... Did you get my idea? Because now I don't have standard mp4 container, I have my own container that is build like AVI (audio video frames one by one in sequence)... – Sirop4ik Sep 16 '19 at 16:35
  • @fadden so, actually I see 2 possible solutions first - somehow adjust extractor to make it know how to read just mp4 chunks and second read needed bytes by yourself and pass this bytes to codec... I think that second solution better... – Sirop4ik Sep 16 '19 at 16:38
  • It sounds like you're defining a custom format for storing video and other data. So you either need your custom format to define where the packet boundaries are, or you need to clone a parser like MediaExtractor to have something that understands how to pick streams out of the .mp4 file, and then chop the stream into NAL units to feed to MediaCodec. Might be easier to retain the .mp4 format and encode your custom data as a separate interleaved stream. Possibly useful information here: https://stackoverflow.com/a/24890903/294248 – fadden Sep 16 '19 at 18:37
  • @fadden I feel that I did not explain my question in the right shape. To make it simpler, the question is: if is it possible to pass byte buffer to the codec (in order to decode), instead of file path or not? This is the question. Because I did not find any information about it, but as for me it should be a trivial task... pass byte buffer to the codec in order to decode... – Sirop4ik Sep 17 '19 at 14:06
  • Something has to identify the NAL units. If you start with a .mp4, that means splitting out the elementary streams, and then parsing the video stream into packets. Normally MediaExtractor does this. MediaExtractor and MediaCodec don't talk to each other; your code passes the output of ME into MC (see e.g. all examples on https://bigflake.com/mediacodec/). If you want to replace ME with your own code that finds NAL units and PTS values, you can. The source code for MediaExtractor is part of AOSP, so you can clone that and modify it how you want. – fadden Sep 17 '19 at 15:33
  • @fadden fadden it is looks odd... I can't belive that solution for my question so complicated. Look, what is ME work - is take filepath and extract and split bytes(by frames as far as I understood). I want to make life easier for ME and read mp4 for him and pass to ME ready-made bytes, so ME should not read the file just split bytes. Something like `AMediaExtractor_setDataSource(ex, myByteBuffer)`. What do you think? – Sirop4ik Sep 19 '19 at 08:23
  • @fadden I found a way to make it :) But I am not sure about it... Edited my question, could you take a look? What do you think? – Sirop4ik Sep 19 '19 at 11:53
  • probably, you will be better served with a custom DataSource, see https://medium.com/@jacks205/implementing-your-own-android-mediadatasource-e67adb070731 – Alex Cohn Oct 28 '19 at 13:50

0 Answers0