Converting Unity texture2d to YUV420P for ffmpeg

Question

I'm trying to create video recorder inside Unity player using FFmpeg.

I have next code snippets:

Unity, C#:

Texture2D frameTexture = new Texture2D(frameWidth, frameHeight, TextureFormat.RGB24, false);
//...
frameTexture.ReadPixels(new Rect(0, 0, frameWidth, frameHeight), 0, 0, false);
frameTexture.Apply();
//.....
byte[] pixels = frameTexture.GetRawTextureData();
AddFrame(pixels, pixels.Length, libApi);


//DLL Function Declaration
[DllImport("VideoCapture", CallingConvention = CallingConvention.Cdecl)]
static extern void AddFrame(byte[] data, int size, System.IntPtr api);

C++ code:

__declspec(dllexport) void AddFrame(uint8_t *data, int size, VideoCapture *vc) {
    vc->AddFrame(data, size);
}

void VideoCapture::AddFrame(uint8_t *data, int size) {
    Log("\nAdding frame\n");

    int err;
    if (!videoFrame) {
        Log("Allocating Video Frame\n");

        videoFrame = av_frame_alloc();
        videoFrame->format = AV_PIX_FMT_YUV420P;
        videoFrame->width = cctx->width;
        videoFrame->height = cctx->height;

        if ((err = av_frame_get_buffer(videoFrame, 32)) < 0) {
            Debug("Failed to allocate picture", err);
            return;
        }
    }

    if (!swsCtx) {
        Log("Creating SWS context\n");
        swsCtx = sws_getContext(cctx->width, cctx->height, AV_PIX_FMT_RGB24, cctx->width, cctx->height, AV_PIX_FMT_YUV420P, 0, 0, 0, 0);
    }

    uint8_t * inData[1] = { data };
    int inLinesize[1] = { 3 * cctx->width };

    Log("Scaling data\n");
    // From RGB to YUV
    if ((err = sws_scale(swsCtx, inData, inLinesize, 0, cctx->height, videoFrame->data, videoFrame->linesize)) < 0) {
        Debug("Failed to scale img", err);
        return;
    }
    ...........

Unity player crashes when it's doing "sws_scale".

Logs from Unity:

  ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF8437B49A6)
0x00007FF8437B49A6 (swscale-4) 
  ERROR: SymGetSymFromAddr64, GetLastError: 'Attempt to access invalid address.' (Address: 00007FF8437B2982)
0x00007FF8437B2982 (swscale-4) 
0x00007FF8437E78A6 (swscale-4) sws_get_class
0x00007FF8437E8E47 (swscale-4) sws_scale

I thought its because of 'sws_get_class', but it works if I call this function directly. It also works if I pass empty data into 'sws_scale', only complaining that it's NULL. So, the reason is in data itself, but I don't what's wrong with it.

I also tried to encode texture to JPG and PNG, pinned array into memory, but the result didn't change.

Thanks in advance

UPD:::

Changed DLL function calling to

unsafe void AddFrame(byte[] data)
{
    fixed (byte* p = data)
    {
        AddFrame((IntPtr)p, data.Length, customLib);
    }
}
//Function declaration
static extern void AddFrame(IntPtr data, int size, System.IntPtr api);

But error is still there.

Your C# `AddFrame` function takes 3 params but your C++ `AddFrame` function has only 2 params....Also, you are not pinning the array. You basically ignore what I said in my [other](https://stackoverflow.com/a/46527921/3785314) answer. — Programmer, Oct 03 '17 at 13:18
@Programmer, I added all missed information to main question. I did all as you said in that answer. — DDovzhenko, Oct 03 '17 at 13:28
Don't remove it in the future even if you are experiencing other problems. If it's not required I wouldn't spend time writing that. as for your problem, I don't really know since most things I had in mind are now eliminated but how do you know that the problem is from `sws_scale` line? — Programmer, Oct 03 '17 at 13:34
@Programmer Sorry for that. I have logging before and after sws_scale and it doesn't fail if I remove this line (just creates empty video with green screen) — DDovzhenko, Oct 03 '17 at 13:36
But do you see the logs such as "Scaling data"? If so, where? — Programmer, Oct 03 '17 at 14:46
@Programmer In file. Log is just a function: void Log(std::string str) { logFile.open("Logs.txt", std::ofstream::app); logFile.write(str.c_str(), str.size()); logFile.close(); } — DDovzhenko, Oct 03 '17 at 14:49
Use [this](https://stackoverflow.com/questions/43732825/use-debug-log-from-c) for C++ when debugging. It will show up in the Editor and save you some time. I looked on the [`sws_scale`](https://www.ffmpeg.org/doxygen/3.1/swscale_8h.html) doc for few minutes but don't know the problem is. Check [this](https://github.com/keijiro/FFmpegOut) project that records video in Unity out. — Programmer, Oct 03 '17 at 16:35
I've finally found the reason. Unity's frameWidth and frameHeight where calculated in runtime and it wasn't required 1920x1080. If I pass correct source height into 'sws_scale', it works good. — DDovzhenko, Oct 04 '17 at 08:17
Good. You can post your solution as answer to help other people in the future. — Programmer, Oct 04 '17 at 08:18

score 1 · Accepted Answer · answered Oct 04 '17 at 08:25

The reason why sws_scale was failing was passing incorrect source Height and Width for sws_getContext and for sws_scale. You should pass dimensions into method or hardcore values in Unity.

Also sws_scale returns height of the output slice, not an error.

Converting Unity texture2d to YUV420P for ffmpeg

1 Answers1