How to encode video with transparent background

Question

I am encoding a video using cocoa for OSX (with AVAssetWriter) in h264. This is the configuration:

// Configure video writer
AVAssetWriter *m_videoWriter = [[AVAssetWriter alloc] initWithURL:[NSURL fileURLWithPath:@(outputFile)] fileType:AVFileTypeMPEG4 error:NULL];

// configure video input
NSDictionary *videoSettings = @{ AVVideoCodecKey : AVVideoCodecH264, AVVideoWidthKey : @(m_width), AVVideoHeightKey : @(m_height) };
AVAssetWriterInput* m_writerInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeVideo outputSettings:videoSettings];

// Add video input into video writer
[m_videoWriter addInput:m_writerInput];

// Start video writer
[m_videoWriter startWriting];
[m_videoWriter startSessionAtSourceTime:kCMTimeZero];

I am using a 'AVAssetWriterInputPixelBufferAdaptor' element to add frames to the composition like this:

AVAssetWriterInputPixelBufferAdaptor *m_pixelBufferAdaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:m_writerInput sourcePixelBufferAttributes:NULL];

uint8_t* videobuffer = m_imageRGBA.data;


CVPixelBufferRef pixelBuffer = NULL;
CVReturn status = CVPixelBufferCreate (NULL, m_width, m_height, kCVPixelFormatType_32ARGB, NULL, &pixelBuffer);
if ((pixelBuffer == NULL) || (status != kCVReturnSuccess))
{
    NSLog(@"Error CVPixelBufferPoolCreatePixelBuffer[pixelBuffer=%@][status=%d]", pixelBuffer, status);
    return;
}
else
{
    uint8_t *videobuffertmp = videobuffer;
    CVPixelBufferLockBaseAddress(pixelBuffer, 0);
    GLubyte *pixelBufferData = (GLubyte *)CVPixelBufferGetBaseAddress(pixelBuffer);

    //printf("Video frame pixel: %d, %d, %d, %d\n", videobuffertmp[0], videobuffertmp[1], videobuffertmp[2], videobuffertmp[3]);

    for( int row=0 ; row<m_width ; ++row )
    {
        for( int col=0 ; col<m_height ; ++col )
        {
            memcpy(&pixelBufferData[0], &videobuffertmp[3], sizeof(uint8_t));       // alpha
            memcpy(&pixelBufferData[1], &videobuffertmp[2], sizeof(uint8_t));       // red
            memcpy(&pixelBufferData[2], &videobuffertmp[1], sizeof(uint8_t));       // green
            memcpy(&pixelBufferData[3], &videobuffertmp[0], sizeof(uint8_t));       // blue

            pixelBufferData += 4*sizeof(uint8_t);
            videobuffertmp  += 4*sizeof(uint8_t);
        }
    }

    CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
}

// transform new frame into pixel buffer
[m_pixelBufferAdaptor appendPixelBuffer:pixelBuffer
                   withPresentationTime:CMTimeMake(m_frameNumber, m_framesPerSecond)];

CFRelease(pixelBuffer);
pixelBuffer = nil;

In the pixel data, the alpha value for the pixels are defined as transparent, but the video has not transparent regions.

I am not sure if the encoder is ignoring the alpha value, or is not possible to encode a video with transparent regions. Is there any way to include the alpha channel values in the encoding process?

A working impl can be found at this answer: http://stackoverflow.com/a/26965602/763355 — MoDJ, Apr 27 '16 at 19:43

score 2 · Accepted Answer · edited May 23 '17 at 12:33

2

You probably can't, at least in H.264. See: How to create an h264 video with an alpha channel for use with HTML5 Canvas?

I guess the transparency can be used in mixing and effects processes before encoding to H.264 but not in the final output.

One workarounds may be setting the transparent areas to a pure green value and using that colour as a matte in a later video editing process (like when they use a green background for the weather forecast). Obviously that only works if the output is intended for such an editing process rather than as final output.

edited May 23 '17 at 12:33

Community

1
1

answered May 18 '14 at 00:19

Joseph Lord

6,446
1
28
32

Yes, you are wright. The encoder ignores the alpha values. I am using a format with alpha layer support. Pro Res 4444 in this case. Thank you. – goe May 19 '14 at 08:13

How to encode video with transparent background

1 Answers1

Linked