0

I'm trying to implement a Gstreamer plugin _chain function based on its template, which simply replaces the input video frame(BGRx) to my converted image in the same size, shown as below:

static GstFlowReturn gst_myplugin_chain (GstPad * pad, GstObject * parent, GstBuffer * buf) {
  Gstmyplugin *filter；
  filter = GST_MYPLUGIN (parent);
  if(filter->silent == FALSE)
     g_print("I'm plugged, therefore I'm in. \n");
  // retrieve pass-in image frame from input buf
  GstMapInfo map;
  gst_buffer_map(buf, &map, GST_MAP_READ);
  cv::Mat frame(cv::Size(width, height), CV_8UC4, (char *)map.data, cv::Mat::AUTOSTEP);      //Q1

  // check the orignal frame
  cv::imshow("Input image", frame);           //Q2

  // convert to new image by same size
  cv::Mat out = my_convert(frame);

  // how to populate output buffer from Mat out then
  GstBuffer *out_buf = ???              //Q3

  //return gst_pad_push(filter->srcpad, buf);   transparent filter in plugin template

  return gst_pad_push(filer->srcpad, out_buf);   //send my converted image
}

So there are three questions(also marked in above lines):

Q1: how to get input image's width and height ?

Q2: why the parsed image can't be shown properly as those in the video ?

Q3: how to populate the output buffer ?

I've searched a lots online for days, but still can't get them solved yet.

nayana
  • 3,787
  • 3
  • 20
  • 51
Ian S
  • 11
  • hi, edited your post a bit so it does not trigger anyone and is more readable. Check this comment - there is just some memcpy which takes data from Mat::data member variable. No idea if its correct. https://github.com/IntelRealSense/librealsense/issues/2634#issuecomment-433904079 And check this comment https://stackoverflow.com/a/10266854/3876138 So when you got the raw data, you can just use GstMap and map the raw data to GstBuffer. – nayana Mar 10 '20 at 14:07
  • Thanks nayana for getting this clarified, those are very helpful for understanding – Ian S Mar 13 '20 at 02:40
  • usually its a good thing to even answer your own question if you have figured it out :) it can help others – nayana Mar 18 '20 at 16:28
  • Q1 - I guess you get this from caps, which you could store when caps event arrives Q2 - what happens visually (window is opened, garbadge is show?) ? I have no idea, maybe the format you used is different than what is in buffer? Q3 - default construct the GstBuffer, map the out_buf buffer with GST_MAP_WRITE and memcpy the data into the map.data – nayana Mar 18 '20 at 16:38

1 Answers1

1

Q1: how to get input image's width and height ?

This is done in the gst_my_filter_sink_event function when it receives the GST_EVENT_CAPS, from which you can get the height/width. Note that I added height and width variables to the filter class.

/* this function handles sink events */
static gboolean
gst_my_filter_sink_event (GstPad * pad, GstObject * parent,
    GstEvent * event)
{
  GstMyFilter *filter;
  gboolean ret;

  filter = GST_MYFILTER (parent);

  GST_LOG_OBJECT (filter, "Received %s event: %" GST_PTR_FORMAT,
      GST_EVENT_TYPE_NAME (event), event);

  switch (GST_EVENT_TYPE (event)) {
    case GST_EVENT_CAPS:
    {
      GstCaps *caps;

      gst_event_parse_caps (event, &caps);
      /* do something with the caps */
      GstStructure *caps_struct = gst_caps_get_structure(caps, 0);
      if( !caps_struct) {
        g_warning("caps have NULL structure");
      }


      if( !gst_structure_get_int(caps_struct, "width", &filter->width) ||
          !gst_structure_get_int(caps_struct, "height", &filter->height)) {
        g_warning("caps have no HEIGHT, WIDTH");
      }

      const gchar* format = gst_structure_get_string(caps_struct, "format");

      g_info("height: %d, width: %d, format: %s", filter->height, filter->width, format);

      /* and forward */
      ret = gst_pad_event_default (pad, parent, event);
      break;
    }
    default:
      ret = gst_pad_event_default (pad, parent, event);
      break;
  }
  return ret;
}
dlwalter
  • 11
  • 1