3

For the sake of continuity, let us assume "RGB values" are the following:

typedef struct RGB {
    uint8_t r, g, b;
} rgb;

However, if you feel that a different color space is more appropriate for this question, please use that instead.

How might I go about writing 2D arrays of RGB values to a video in C given an output format and framerate?

Before I continue, I should specify that I wish to be able to do this all within one program. I am trying to add functionality to an application that would allow it to compile videos frame by frame without having to leave it.

Additionally, my needs for this functionality are extremely basic; I simply need to be able to set individual pixels to certain colors.

The closest I have come to a solution so far is the C library FFmpeg. Allow me to describe what I was able to learn on my own:

After looking through its documentation, I came across the function avcodec_send_frame(avctx, frame), whose parameters are of the types AVCodexContext* and const AVFrame* respectively. If these are not the right tools for what I am trying to do, please ignore the rest of the question and instead point me towards what I should be using.

However, I do not know which fields of avctx and frame must be set manually and which do not. The reason I assume some do not is because both are extremely large structures, but correct me if I am wrong.

Question 1: What values of an AVCodecContext and AVFrame must be set? Of these, what is/are the recommended value(s) for each of them?

Additionally, I was only able to find instructions on how to initialize an AVFrame (using av_frame_alloc() and av_frame_get_buffer()) but not for an AVCodexConstant.

Question 2: Is there a proper way to initialize an AVCodexConstant? And just in case, is the method of initializing an AVFrame described above correct? Do any of the fields of either have a proper method of initialization?

Also, I was not able to find official documentation on how to take this AVCodexConstant (which I assume contains the video information) and turn it into a video. I apologize if the documentation for this is easy to find and I just missed it.

Question 3: How do I turn an AVCodexConstant into a file of a given format?

And, given my limited knowledge:

Question 4: Are there any other parts to this process that I am missing, and do I have any of the above parts wrong?

Please keep in mind that I found out about FFmpeg for the first time very recently, and as a result, I am a complete beginner to this. Additionally, my experience with C is very limited, so I would greatly appreciate it if you could note which files need to be included with #include.

Feel free to even go as far as recommending something other than FFmpeg, just as long as it is written in C. I do not need power-user options, but I would greatly prefer flexibility in what audio and video file types the library can handle.

Addressing Potential Duplicates

I appologize for how long this section is; I just want to have my bases covered. I heavily apologize, however, if this is in fact a duplicate of a question that I was just unable to find.

  • ffmpeg C API documentation/tutorial [closed] — This question was too open-ended and received answers pointing the asker towards a tutorial at dranger.com, a tutorial that confusingly muddied the waters by focusing heavily on a graphics library of choice. Please do not take this as me saying it is bad; I am just enough of a beginner that I could not wade through it all.
  • Encoding frames to video with ffmpeg — Although this question seems to have been asking the same thing, it is geared towards Unreal Engine 4, and the asker provided sample code, making it difficult for me to understand which of parts of the accepted answer were necessary for me and which were not.
  • How to write frames to a video file? — While this also asked the same thing, the accepted answer simply provides a command instead of an explanation of code.
  • YUV Raw frames to video stream — While the accepted answer for this question is a command, the question states that it is looking for a way to encode frames generated by C++ code. Is there some way to run commands in code that I haven't been able to find?
  • Converting sequenced frames to video — Not only is the asker's code written in Python, but it also seems to use already-existing image files as frames.
  • How to write bitmaps as frames to H.264 with x264 in C\C++? — The accepted answer seems to describe a process that would take multiple applications, but I could be wrong as I am enough of a beginner that I am not sure exactly what it means other than Step 3.
  • How to write bitmaps as frames to Ogg Theora in C\C++? — Although it isn't a problem that the question specifies the ogg format, it is a problem that the accepted answer suggests libtheora, which appears to only work with ogg files.
Community
  • 1
  • 1
NetherGranite
  • 1,940
  • 1
  • 14
  • 42
  • Here is an example of ffmpeg API usage: [muxing.c](https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/muxing.c) You can see how the AVFrame is filled in `fill_yuv_image().` – GalaDOS Jul 24 '18 at 03:13
  • @GalaDOS Unfortunately, that code appears to use `avcodec_encode_audio2()` and `avcodec_encode_video2()`, which the official FFmpeg documentation says are deprecated and have been replaced by `avcodec_send_frame()`, the function I mentioned in my question. It is useful to see that one can write to an `AVFrame` using its `data` field, but the code writes arbitrary values to `data` in order to produce a "dummy image", so I cannot quite gather anything beyond the fact that `data` is used. Thank you for sharing that, though. – NetherGranite Jul 24 '18 at 03:41

1 Answers1

0

Check this tool bmdplay.cpp This is a linux tool written specifically for certain brand PCI-E cards. Just ignore card specific calls and functions.

it's main() function has answers for questions 1-4. And its Player::ScheduleNextFrame() function does decoding frames part via new API.

What you really asking is as far as I understand to overlay a (small) picture to existing video frame.

Almost always you'll found video files uses YUV format video frames. Most command one is YUV420P. You can convert each frame to RGB then work on it but this will make extra overhead. So it's better use native format.

Key point here is the picture and video frame should have same format. So you should convert the picture to the video frame format beforehand (I assume it's still picture).

RGB is linear format, so if your source happens to be this format, just write an algorithm to copy picture to the video frame. (that is AVFrame::data[0])

YUV420P is planar format. Consist of 3 planes:
Y (AVFrame::data[0])
U (AVFrame::data[1])
V (AVFrame::data[2])
and linesizes (also know as strides) can be obtained from (AVFrame::linesize[0-2])

This format is tricky U and V planes has 1/4 size of Y frame. But this won't be a problem since you'll use same format on the picture as in video frame.

Hope these helps.

the kamilz
  • 1,860
  • 1
  • 15
  • 19