3

I am developing a webrtc simulator in Linux environment using C language. Here I need to send media from one webrtc simulator to other webrtc simulator. I need to work with VP8 and Opus codec. Now I am starting with VP8 codec. As I am new to VP8, kindly help me to understand more as well .

  1. Since it is simulator, I do not want to capture / play media real time. I need to read audio / video data from a file and send it to remote. At remote get the media data extracted, save it to a file.

  2. Which file format contains encoded VP8 data? Is it webm file format or something else?

  3. Assuming webm file contains VP8 and OPUS codec data (which I am not sure), how can I parse frame by frame. For example if I read Extract audio frames from AMR-NB file I can parse amr frames from a file. Similarly is there any help page, where I learn parse logic for VP frame.

  4. Or is there any open source using that I can parse VP8/OPUS frames from a webm file.

Community
  • 1
  • 1
Austin
  • 1,709
  • 20
  • 40
  • 1
    Yes it is webM. Look at this http://www.webmproject.org/code/ – Michael P Jun 16 '15 at 04:17
  • Michael, thanks for input. I checked libwebm. It has few executables like sample, sample_muxer, dumpvtt. Which one I should follow , is it sample.cpp I need to check and understand. Looks sample.cpp explains Matroska parser. But I am looking for VP8 / Opus parser. How Matroska is related to VP8 and OPUS. Kindly let me know. – Austin Jun 16 '15 at 06:12

1 Answers1

4

Q2: I'll start with question number 2, VP8 corresponds with WebM format.

Q3: If you want to parse frame by frame, you need to know the structure of VP8 and OPUS. I don't know what simulation means in your scenario, but in WebRTC environment media data is transfered via RTP. If you are doing simulation without RTP, please start from step 2, otherwise start from No. 1.

  1. Depacketize RTP payload from RTP packet. RTP payload contains VP8 packet info like sequence number, timestamp, etc.
  2. Note, that VP8 frame may consist of multiple VP8 packets, so basically you need to extract the frame content and concatenate it together. There are multiple ways to find out if frame consists of multiple packets, not all of them being fully standardized, but for now I would recommend you to use sequence number, as packets of one frame share the same SEQ.
  3. Write the frame into file. If you want to create a valid WebM file, use function such as av_write_frame() from LibAV (which I personally use).

I can recommend you a piece of code that actually converts raw RTP VP8 packets to VP8 frames: https://github.com/meetecho/janus-gateway/blob/master/postprocessing/pp-webm.c . Packet header reading is used in functions (preprocess and process).

Q4: I use LibAV for such purposes (alternatively FFMPEG).

I'll gladly answer any other question.

Community
  • 1
  • 1
golstar
  • 56
  • 1
  • 7
  • thanks. I will be using RTP. My scenario is something like, from simulator1 call simulator2. Simulator1 will read a webm file, get audio and video data and send to Simulator 2. So first I need to read a webm file, parse VP8 or OPUS frame, send to Simulator 2. Simulator2 on getting VP8/OPUS data will write to a file. I assume webm file contains vp8 and opus encoded data (not raw data). So I can read vp8 and opus encoded data from webm file and send it to simulator2. Is there any example code I can refer. Above link will be useful at simulator2 side while writing data to file. – Austin Jun 16 '15 at 16:37
  • For reading I would use av_read_frame, I found this example. Your source will be WebM file of course, not real-time stream (RTSP). http://hasanaga.info/tag/ffmpeg-libavcodec-av_read_frame-example/ – golstar Jun 17 '15 at 08:07
  • This example looks much better: https://ffmpeg.org/doxygen/2.1/doc_2examples_2demuxing_8c-example.html What should be interesting for you is av_read_frame and av_decode_video2 functions. – golstar Jun 17 '15 at 08:14
  • First built libav-10.7 and next built the source given in https://ffmpeg.org/doxygen/2.1/doc_2examples_2demuxing_8c-example.html. Got some link errors (for av_get_media_type_string(), av_ts2timestr(), any idea why it is so..). Since these were used in printf/fprintf so I just commented these lines and built successfully. Downloaded a sample webm file (vp8/vorbis) and was able to generate raw audio and video files. Logs during execution says play raw audio by using ffplay. Where can I get ffplay (for Ubuntu). Please let me know. – Austin Jun 18 '15 at 02:30
  • This is the thing, some time ago LibAV and FFMPEG forked. You can read about this here: http://stackoverflow.com/questions/9477115/what-are-the-differences-and-similarities-between-ffmpeg-libav-and-avconv . For LibAV there is avplay and for FFMPEG there is ffplay. Do you use FFMPEG or LibAV? – golstar Jun 18 '15 at 07:42
  • I have taken LibAV. I tried both libav-10.7 and libav-11.4 from https://libav.org/download.html page. I built (configure, make and make install) both libav. But I never got avplay. Do I need to install it separately. – Austin Jun 18 '15 at 12:47
  • I installed libav-tools package in my ubuntu m/c. Then avplay was available. I played video using "avplay -f rawvideo -video_size 560x320 small-video.raw" and audio using " avplay -f f32le -ac 1 -ar 48000 small-audio.raw" command. I got expected result. Thanks a lot. Now the next question is which one you would suggest out of ffmpeg or libav. For me I will choose that one which takes less resource (cpu and memory). Keeping low usage of cpu and memory which one you recommend between ffmpeg and libav. – Austin Jun 18 '15 at 13:20
  • Algorithms SHOULD be pretty similar for both, because they came from the same base. However I believe, that there is a reason why LibAV is supported as an official Ubuntu package. This is still pretty subjective, since I use LibAV. – golstar Jun 18 '15 at 14:08
  • It has helped me to great extent. I accept your answers (sorry, is there any other way to accept it). I tried up-voting, but it is declining telling I need to have 15 score to do that. I am coming up with a more clear question (still bit struggling), kindly help me with that. – Austin Jun 25 '15 at 01:52
  • Hi, I have a more clear question (bit different than this) at http://stackoverflow.com/questions/31040600/webm-vp8-opus-file-read-and-write-back , can you kindly check and guide me. – Austin Jun 25 '15 at 03:16
  • Thank you. This is how to accept an answer: http://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work . It will give 2 rep points to you also. I'll look into your next question in few hrs. – golstar Jun 25 '15 at 07:43