how to convert AVFrame to texture used by glTexImage2D?

Question

as you know that AVFrame has 2 property:pFrame->data, pFrame->linesize. After i read frame from video /sdcard/test.mp4 (android platform), and convert to RGB AVFrame vice:

img_convert_ctx = sws_getContext(pCodecCtx->width, pCodecCtx->height, 
                   pCodecCtx->pix_fmt, 
                   target_width, target_height, PIX_FMT_RGB24, SWS_BICUBIC, 
                   NULL, NULL, NULL);
            if(img_convert_ctx == NULL) {
                LOGE("could not initialize conversion context\n");
                return;
            }
            sws_scale(img_convert_ctx, (const uint8_t* const*)pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);

I get pFrameRGB after converted. I need to texture it in opengl by using glTextImage2D:

// Create The Texture
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 256, 256, 0, GL_RGB, GL_UNSIGNED_BYTE, data);

// i don't know data in here is pFrameRGB->data or not,if not, how to convert it to approriate format for glTextImage so i can display video in opengl, using AVFrame and gltextImage2D. All helps from u are very appreciated.

score 1 · Answer 1 · answered Apr 05 '11 at 13:42

1

The actual pixel data for the pFrameRGB is in pFrameRGB->data[0]... In your glTexImage2D call use GL_RGBA instead of GL_RGB. Thats what I did to get it working anyway. Assuming you have your texture and vertex coordinates created correctly then that should get it working. I can post some code examples if that doesn't help.

answered Apr 05 '11 at 13:42

DRiFTy

11,269
11
61
77

Can you post some code example using pFrameRGB->data[0] in GL_RGBA to texture, i also do that as: sws_scale(img_convert_ctx, (const uint8_t* const*)pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize); data=(char*)malloc(pFrameRGB->linesize[0]*target_height) memcpy(data,pFrameRGB->data,pFrameRGB->linesize[0]*target_height); *w=320,*h=240; my function return data, widht and height to glTextImage2D: glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, *w, *h, 0, GL_RGBA, GL_UNSIGNED_BYTE, data); but i don;t get any appearing image, am i wrong – SieuTruc Apr 07 '11 at 18:02
Hi, could you post some code examples Kieran? I'm just getting a blank screen, in my SwsContext I'm changing the 854x480 input to be converted to a RGB24 at 512x256 (for the power of 2 requirement) then in sws_scale() I convert and put in pFrameRGB and then my glTexImage2D looks like this: glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 512, 256, 0, GL_RGBA, GL_UNSIGNED_BYTE, pFrameRGB->data[0]); but I'm getting nothing, completely blank screen. Any ideas? – Infiniti Fizz Jun 27 '11 at 15:04

score 1 · Answer 2 · answered Jun 13 '11 at 22:38

Two things: (1) is target_width and target_height powers of 2? If not, you'll get a blank screen. You need to scale the picture to a power of 2. (2) where it says "data" in glTexImage, you need to use pFrameRGB->data[0] as Kieran pointed out.

One more thing, you don't need to use RGBA (unless it's faster. I haven't tested it out). If you do, you change the ffmpeg target format to PIX_FMT_RGBA and then change both instances of GL_RGB to GL_RGBA.

Other than that, I don't see anything wrong with your code. I have almost the same exact code and it runs just fine.

how to convert AVFrame to texture used by glTexImage2D?

2 Answers2

Linked