2

I want to extract the current image from the FFMPEG standard output and show it on a C# form. The stream source itself is a h264 raw data which converted into image and piped to the standard output. Here is my code, but I have no idea how to process the output (maybe MemoryStream):

public Process ffproc = new Process();
private void xxxFFplay()
{
    ffproc.StartInfo.FileName = "ffmpeg.exe";
    ffproc.StartInfo.Arguments = "-y -i udp://127.0.0.1:8888/ -q:v 1 -huffman optimal -update 1 -f mjpeg -";

    ffproc.StartInfo.CreateNoWindow = true;
    ffproc.StartInfo.RedirectStandardOutput = true;
    ffproc.StartInfo.UseShellExecute = false;

    ffproc.EnableRaisingEvents = true;
    ffproc.OutputDataReceived += (o, e) => Debug.WriteLine(e.Data ?? "NULL", "ffplay");
    fproc.ErrorDataReceived += (o, e) => Debug.WriteLine(e.Data ?? "NULL", "ffplay");
    ffproc.Exited += (o, e) => Debug.WriteLine("Exited", "ffplay");
    ffproc.Start();

    worker = new BackgroundWorker();
    worker.DoWork += worker_DoWork;
    worker.WorkerReportsProgress = true;
    worker.ProgressChanged += worker_ProgressChanged;
    worker.RunWorkerAsync();
}

public void worker_DoWork(object sender, DoWorkEventArgs e)
{
    try
    {
        var internalWorker = sender as BackgroundWorker;
        Process p = e.Argument as Process;
        buffer = new MemoryStream();
        bufferWriter = new BinaryWriter(buffer);
        using (var reader = new BinaryReader(p.StandardOutput.BaseStream))
        {
            while (true)
            {
                bufferWriter.Write(1);
                var img = (Bitmap)Image.FromStream(buffer);
                pictureBox1.Image = img;
                //get the jpg image
            }
         }
     }
     catch (Exception ex)
     {
           // Log the error, continue processing the live stream
     }
 }

Any help would be appreciated!

Alexander Streckov
  • 21
  • 2
  • Why on earth would you use standard output as a means to input _binary_ data? –  Jan 30 '18 at 08:26
  • Ok. Then how should I read the data? – Alexander Streckov Jan 30 '18 at 08:39
  • Use an API not some fiddly piddily console app. Why not take a look at _Media Foundation_ or _DirectShow_? Wishing you well –  Jan 30 '18 at 08:41
  • 3
    @MickyD: processing stdout as binary is very common, even on Windows. Programs like ffmpeg, gzip, and others which handle binary data through pipes all work that way. – Peter Duniho Jan 30 '18 at 08:45
  • Those APIs cannot provide me low latency video streaming over the network (low latency means <100ms for me...) And using ffmpeg encoding I could send (with appropriate settings) live video stream. – Alexander Streckov Jan 30 '18 at 08:47
  • My only problem is how to catch the data, recognize as an image (eg. header or something) and show it in a picturebox. – Alexander Streckov Jan 30 '18 at 08:50
  • FWIW, I doubt ffmpeg will provide better latency than DirectShow. But, if you insist, dealing with stdout as binary data is straight-forward, just use `StandardOutput.BaseStream`, per marked duplicates. – Peter Duniho Jan 30 '18 at 08:50
  • Ok, thanks Peter. Second part of my question is how to identify the jpeg in the stream? `using (var reader = new BinaryReader(p.StandardOutput.BaseStream)) { while (true) { //get the jpg image } }` I mean according the duplicate post, the 'reader' variable contains the actually read bytes, but that might contain the last part of the previous jpg and the first part of the next jpg, not? – Alexander Streckov Jan 30 '18 at 11:01
  • _"Those APIs cannot provide me low latency video streaming over the network..."_ - doubt that. DirectShow has been around for eons and is used in MS Media Server and Media Center –  Jan 30 '18 at 19:51
  • _"Second part of my question is how to identify the jpeg in the stream?"_ -- Stack Overflow is not conducive for, nor tolerant of, multiple questions in the same post. Now that you know not to use `OutputDataReceived`, you can work on interpreting the data. That said, keep in mind that given ffmpeg has no actual programming API, any question that simply asks about the format of the data is _not_ a programming question and would be off-topic for Stack Overflow. – Peter Duniho Jan 31 '18 at 01:33
  • How are you going to decipher what part in the random data pipe stream constitutes the beginning of a new frame? The time spent mucking about with raw data either looking for the frame or waiting for it to appear rather than using a professional video codec API will cost you latency wise. You're essentially re-inventing a video codec decoder most likely with zero hardware support –  Jan 31 '18 at 07:19
  • Winforms is not hardware accelerated by the way –  Jan 31 '18 at 07:42
  • @PeterDuniho Well true, but it's unlikely though for extracting a _random frame out of a live video stream_. It puts the burden of decoding the video (or _"pars[ing]"_ as Bradley says in the general sense) into the client app, in this case a c# software-driven video decoder and presenter app that has to do at least 10 FPS as inferred by the OP. –  Jan 31 '18 at 08:04

0 Answers0