1

I've got a problem similar to the question/answer posed here: https://stackoverflow.com/a/22461014. However, the difference is that the I'm trying to decode from the UnityMain thread (Unity's main loop). Calling from Update(), I pass a byte array and a textureId to MediaCodec's decoder. 

public void decodeFrameToTexture(BytePointer pixels, int len, int textureID) {
    if ( this.textureID != textureID) {
        Log.d(TAG, "TextureID changed: " + textureID);
        this.textureID = textureID;

        SurfaceTexture surfaceTexture = new SurfaceTexture(textureID);
        mSurface = new Surface(surfaceTexture);

        outputSurface = new CodecOutputSurface(width, height, textureID);
    }

... then we do the decoding (basically a copy of the code at http://bigflake.com/mediacodec/ExtractMpegFramesTest.java.txt but without the while loop, as this is frame-by-frame. Also copied the CodecOutputSurface and supporting classes basically verbatim).

Finally we have this code: 

 decoder.releaseOutputBuffer(decoderStatus, info.size != 0 && outputSurface != null /*render*/);

        if ( outputSurface != null ) {
            outputSurface.awaitNewImage();
            outputSurface.drawImage(true);
        }

The trouble is, awaitNewImage() always times out without getting a frame, leading back to the problem referenced here, that the onFrameAvailable() callback is never getting called.

For reference, UnityMain does not have a Looper component. When running this code: 

    Looper looper;
    if ((looper = Looper.myLooper()) != null) {
        mEventHandler = new EventHandler(looper);
    } else if ((looper = Looper.getMainLooper()) != null) {
        mEventHandler = new EventHandler(looper);
    } else {
        mEventHandler = null;
    }

the assigned looper from that thread is the MainLooper looper. Any ideas would be appreciated. As stated by @fadden, "the trick is to make sure that frame-available events arrive on a different thread from the one sitting in awaitNewImage()". Given that we're running 

mSurfaceTexture.setOnFrameAvailableListener(this);

from UnityMain, I think this satisfies this requirement? The callback should be called from the "main" thread?

Community
  • 1
  • 1
Scott Montgomerie
  • 1,780
  • 1
  • 16
  • 17
  • In the other question, the frame-available callback always fires "late", i.e. after the timeout, because the "frame available" message is arriving in the Looper on the thread that's waiting to be signaled. It needs to arrive in a Looper on a thread that's looping. If it's *never* getting called, then you may have a different problem (e.g. your frames aren't making it to the SurfaceTexture). – fadden Mar 24 '15 at 15:25

0 Answers0