Suppose you need to save the game play as video file or you want to live stream the game, as far as I know there is no simple way to do this with blueprints nor there are plugins for this purpose. To solve this you need to somehow get the video and audio of game play. After some research I found a way to get the video of game play from Viewport. You can achieve this by inheriting from UGameViewportClient and overriding Draw method.
void UMyGameViewportClient::Draw(FViewport* Viewport, FCanvas* SceneCanvas)
You can call ReadPixel on Viewport to get an RGBA representation and encode this color buffer using FFmpeg library.
TArray<FColor> ColorBuffer;
if (!Viewport->ReadPixels(ColorBuffer, FReadSurfaceDataFlags(),
FIntRect(0, 0, ViewportSize.X, ViewportSize.Y)))
{
UE_LOG(LogTemp, Error, TEXT("Cannot read from viewport"));
return;
}
Here is the answer to question Encoding frames to video with ffmpeg
So far so good. Now what about audio? Which class in Unreal Engine is responsible for mixing all audios to get a single, final audio to be played? How to get the audio from that class?
