3

I am a beginner with gstreamer so bear with me.

I have a working pipeline where audio and video from a test source is sent to the webrtcbin element used to send out offer. Pipeline is as follows:

PIPELINE_DESC = '''
webrtcbin name=sendrecv stun-server=stun://stun.l.google.com:19302
 audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay !
 queue ! application/x-rtp,media=audio,encoding-name=OPUS,payload=96 ! sendrecv.
 videotestsrc is-live=true pattern=ball ! video/x-raw,width=320,height=240 ! videoconvert ! queue ! x264enc ! rtph264pay !
 queue ! application/x-rtp,media=video,encoding-name=H264,payload=97 ! sendrecv.
'''

However doing this consumes a lot of CPU/Memory as gstreamer has to encode audio/video. Hence I was to use a pre-recorded file to lower the resource usage.

I want to use a sample file (sample.mp4) to send audio and video to the webRTCbin element. The mp4 file has H264 video and AAC audio. I have tried a lot of combinations of elements but it is not working. Could you please help me correct my pipeline? 

PIPELINE_DESC = '''
webrtcbin name=sendrecv stun-server=stun://stun.l.google.com:19302
 filesrc location=sample.mp4 ! decodebin ! audioconvert ! sendrecv.
 filesrc location=sample.mp4 ! decodebin ! videoconvert ! sendrecv.
'''

Many thanks in advance.

Mayank
  • 51
  • 5

1 Answers1

3

mp4 file is a container file format and it needs to be demultiplexed to get video and audio. For that purpose, you can use GStreamer's qtdemux element.

enter image description here

Considering above, example pipeline could be something like this

PIPELINE_DESC = '''
filesrc location=test.mp4 ! qtdemux name=demux
webrtcbin name=sendrecv stun-server=stun://stun.l.google.com:19302
demux.audio_%u ! aacparse ! rtpmp4apay !
queue ! application/x-rtp,media=audio,encoding-name=MP4A-LATM,payload=96 ! sendrecv.
demux.video_%u ! h264parse ! rtph264pay !
queue ! application/x-rtp,media=video,encoding-name=H264,payload=97 ! sendrecv.
'''
Dusan Kovacevic
  • 1,377
  • 1
  • 13
  • 19
  • Hi Dusan, thanks for your inputs. I changed %u to 0 and the above pipeline works. However, I want an infinite stream of audio/video sent to the sendrecv sink, so is it possible to loop through the test.mp4 to generate the infinite stream. Also, the above pipeline still consumes good amount of CPU. What is causing this, is it the demuxing or the rtpmp4apay/rtph264pay ? If its the demuxing, can it be avoided by using pre-demuxed test files eg. test.aud and test.vid? Could you please share an eg. of such a pipeline which includes looping and doesn't consume much of CPU. Thanks. – Mayank Jan 14 '20 at 20:01
  • Correction, with demux.audio_0 and demux.video_0 only video is working. Audio is not working. Please inform what value needs to used for %u for audio and video. – Mayank Jan 14 '20 at 21:27
  • @Mayank 1. Looping - try [this](https://stackoverflow.com/questions/6833147/looping-a-video-with-gstreamer-and-gst-launch) SO Q/A , 2. Cannot help much with CPU usage since many factors can affect it. Try to change webrtcbin with some other sink, try to decrease video resolution...and observe. 3. What does it mean "audio not working"? Provide some more details (e.g. Gstreamer debug, etc...) – Dusan Kovacevic Jan 15 '20 at 07:36
  • I use this pipeline string and the browser throws an error: "can't set remote descriptor". Any idea? – Jim Jin Jun 13 '23 at 10:43