GStreamer freezes at first frame

Question

I wish to split the GStreamer pipeline so the video is both displayed and recorded. I got this working with gst-launch using the following pipeline:

gst-launch-1.0 -v videotestsrc ! video/x-raw,width=640,height=480 tee name="t" ! queue ! glupload ! glimagesink t. ! queue ! jpegenc ! avimux ! filesink location=output.avi

I've tried to implement this in my C++ application, but ran in to a problem that doesn't make sense to me. Below is the entire pipeline setup, but only the last couple of lines are relevant/interesting.

gst_init (NULL, NULL);

GstElement *pipeline = gst_pipeline_new(NULL);
GstElement *sink = NULL;

GstElement *src = gst_element_factory_make("videotestsrc", NULL);
g_assert(src);

GstElement *filter = gst_element_factory_make("capsfilter", "filter");
g_assert(filter);

g_object_set(G_OBJECT (filter), "caps", gst_caps_new_simple("video/x-raw", 
    "width", G_TYPE_INT, 640,
    "height", G_TYPE_INT, 480,
    NULL), 
NULL);

GstElement *convert = gst_element_factory_make("videoconvert", NULL);
g_assert(convert);

// Tee
GstElement *tee = gst_element_factory_make("tee", "videotee");
g_assert(GstElement *tee);

// Display queue
GstElement *displayQueue = gst_element_factory_make("queue", "displayQueue");
g_assert(displayQueue);


GstElement *upload = gst_element_factory_make("glupload", NULL);
g_assert(upload);


sink = gst_element_factory_make("qmlglsink", NULL);
g_assert(sink);

// Record queue
GstElement *recordQueue = gst_element_factory_make("queue", "recordQueue");
g_assert(recordQueue);

GstElement *encode = gst_element_factory_make("jpegenc", NULL);
g_assert(encode);

GstElement *mux = gst_element_factory_make("avimux", NULL);
g_assert(mux);

GstElement *filesink = gst_element_factory_make("filesink", NULL);
g_assert(filesink);

g_object_set(G_OBJECT(filesink), "location", "output.avi", NULL);

// The above is not interesting, just included it for completeness

// Add elements to bin
gst_bin_add_many(GST_BIN (pipeline), src, filter, convert, tee, displayQueue, upload, sink, recordQueue, encode, mux, filesink, NULL);

// Link elements
gst_element_link_many(src, filter, convert, tee, NULL);
gst_element_link_many(tee, displayQueue, upload, sink, NULL);
//gst_element_link_many(tee, recordQueue, encode, mux, filesink, NULL);

If I don't add filesink to the bin, then the test video is displayed as intended. If I add filesink to the bin, then the video displays the first frame and then freezes. I don't understand why this is, since I haven't linked the filesink to the pipeline yet (the line is commented out).

Does anyone know why this is?

(If I uncomment the line where the record queue is linked to the pipeline, then nothing is displayed. But this might be a question for later.)

UPDATE

I tried to run the pipeline using gst_parse_launch() and it shows the same behaviour.

GError *error = NULL;
GstPipeline *pipeline;
pipeline = GST_PIPELINE(gst_parse_launch("videotestsrc  ! video/x-raw,width=640,height=480 tee name='t' ! queue ! glupload ! glimagesink t. ! queue ! jpegenc ! avimux ! filesink location=output.avi", &error));
gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING);

The code above freezes at first frame and the output.avi file gets created but has a size of 0 bytes. If I remove the filesink queue, then the video displays fine.

Works:

videotestsrc ! video/x-raw,width=640,height=480 tee name="t" ! queue ! glupload ! glimagesink

Doesn't work:

videotestsrc ! video/x-raw,width=640,height=480 tee name="t" ! queue ! glupload ! glimagesink t. ! queue ! jpegenc ! avimux ! filesink location=output.avi

So I guess I have an issue with the filesink plugin. I've tried to use absolute path to file, but that didn't help. (It still works fine when I use gst-launch from terminal)

I've also tried to print some debug information with the environment variable GST_DEBUG="filesink:9", but it doesn't print any errors, only this info:

0:00:00.116304411 3331 0x7f92b4002f60 INFO filesink gstfilesink.c:301:gst_file_sink_set_location: filename : /some/path/output.avi

0:00:00.116345776 3331 0x7f92b4002f60 INFO filesink gstfilesink.c:302:gst_file_sink_set_location: uri : file:///some/path/output.avi

0:00:00.118291875 3331 0x7f92b4002f60 DEBUG filesink gstfilesink.c:523:gst_file_sink_do_seek: Seeking to offset 0 using fseeko

0:00:00.118320893 3331 0x7f92b4002f60 DEBUG filesink gstfilesink.c:423:gst_file_sink_open_file: opened file /some/path/output.avi, seekable 1

Does anyone know what could be causing this?

Answer 1

You could first go with easy option that is use gst_parse_launch(), instead you creating elements and connecting the pads.

Coming to your issue where it freezes, the linking of elements is not correct it needs to be following:

 // Link elements
gst_element_link_many(src, filter, convert, tee, NULL);

/* Manually link the Tee, which has "Request" pads */
  GstPad *tee_display_pad = gst_element_get_request_pad (tee, "src_%u");
  GstPad *queue_display_pad = gst_element_get_static_pad (displayQueue, "sink");

  GstPad *tee_video_pad = gst_element_get_request_pad (tee, "src_%u");
  GstPad *queue_video_pad = gst_element_get_static_pad (recordQueue, "sink");

  if (gst_pad_link (tee_display_pad, queue_display_pad) != GST_PAD_LINK_OK ||
      gst_pad_link (tee_video_pad, queue_video_pad) != GST_PAD_LINK_OK)
    return -1;

  gst_object_unref (tee_display_pad);
  gst_object_unref (queue_display_pad);
  gst_object_unref (tee_video_pad);
  gst_object_unref (queue_video_pad);

  gst_element_link_many(displayQueue, upload, sink, NULL);

gst_element_link_many(recordQueue, encode, mux, filesink, NULL);