22

I would like to listen an RTP audio stream, however the voice has little gaps in it - not continues. What may be the solution? Am I missing something on Receiver(android) side or Streamer(ffmpeg) side?

I'm using ffmpeg to stream RTP audio, 

ffmpeg -f lavfi -i aevalsrc="sin(400*2*PI*t)" -ar 8000 -vcodec pcm_u8 -f rtp rtp://192.168.0.15:41954 (port changes.)

And here is my related android code: 

AudioStream audioStream;
AudioGroup audioGroup;
@Override
public void onStart() {
    super.onStart();
    StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitNetwork().build();
    StrictMode.setThreadPolicy(policy);
    AudioManager audio = (AudioManager)getSystemService(AUDIO_SERVICE);
    audio.setMode(AudioManager.MODE_IN_COMMUNICATION);
    audioGroup = new AudioGroup();
    audioGroup.setMode(AudioGroup.MODE_ECHO_SUPPRESSION);
    InetAddress inetAddress;
    try {
        inetAddress = InetAddress.getByName("192.168.0.15");
        audioStream = new AudioStream(inetAddress);
        audioStream.setCodec(AudioCodec.PCMU);
        audioStream.setMode(RtpStream.MODE_NORMAL);
        InetAddress inetAddressRemote = InetAddress.getByName("192.168.0.14");
        audioStream.associate(inetAddressRemote, 6000);
        ((TextView)findViewById(R.id.tv_port)).setText("Port : " + String.valueOf(audioStream.getLocalPort()));
        audioStream.join(audioGroup);
    }
    catch ( UnknownHostException e ) {
        e.printStackTrace();
    }
    catch ( SocketException e ) {
        e.printStackTrace();
    }
}
Mycoola
  • 1,135
  • 1
  • 8
  • 29
totten
  • 2,769
  • 3
  • 27
  • 41
  • Have you tried opening the RTP stream in VLC to check its not playing as it was sent (data loss). this would also allow you to Wireshark the network traffic and i believe VLC will let you stream it back out allow you to connect your device to that and see if it different to the received stream on the pc – Shaun07776 Dec 15 '14 at 20:46
  • Yeah I check it. The audio sound is good with VLC. – totten Dec 16 '14 at 18:47
  • And if you resend it out from vlc and get your device to connect to that is that any better – Shaun07776 Dec 16 '14 at 20:25
  • No better. same result. – totten Dec 16 '14 at 20:38
  • What if you use vlc on the device. Can you monitor the traffic from the server to the device if you can get vlc or another player on the device that may help rule out ffmeg as the issue leaving connectivity issues – Shaun07776 Dec 16 '14 at 23:04
  • Any reason you are using `AudioManager.MODE_IN_COMMUNICATION` and `AudioGroup.MODE_ECHO_SUPPRESSION` in the AudioGroup? – Jeffrey Mixon Dec 17 '14 at 03:42
  • Those MODEs is not changing anything. Tried all configurations. Let me use vlc on the device. – totten Dec 17 '14 at 06:47
  • Cannot listen rtp stream with VLC. No sound. – totten Dec 17 '14 at 07:03
  • Look at the answer I posted. It's android bug, sadly. – totten Dec 17 '14 at 09:17
  • Thank you for your code. I have used this android code to send voice data to vlc player (another android phone). However, I am not able to receive any voice data from vlc streaming (which you have done). I cannot fix this problem. Am I missing some permission? I have used this permission. android:minSdkVersion="17" Thank you. – user1850484 Oct 07 '15 at 19:49
  • it's not a good way to ask a question in comment, just ask a question and post link here instead of veeeery long comment. I'll look at it. – totten Oct 09 '15 at 00:01

2 Answers2

7

Answering my own question, the problem was with android rtp packet management.

Android said that ... assume packet interval is 50ms or less. in the AudioGroup source file.

However, RTP packets are sending with interval 60ms.

That means 50ms is not enough and this leads the problem as described below.

Incoming: X X X X X X Y Y Y Y Y Y X X X X X X Y Y Y Y Y Y X X X X X X
Reading : X X X X X Y Y Y Y Y X X X X X Y Y Y Y Y X X X X X Y Y Y Y Y
          ^ ^ ^ ^ ^ - - - - - - - - - - - - - - - - - - - - ^ ^ ^ ^ ^ 
          ^                                                 ^
          |                                                 |
          |---- just these overlapping packets is valid ----|
          |---- and other packets discarding due to --------|
          |---- invalid RTP headers. -----------------------|

X, Y < packets

I have just one packet in every 300ms interval. That results jittery sound.

I'll send a bug report for this, hope it helps someone.

For ones who are really want to listen raw RTP stream, I suggest them to read packets manually and decode it to PCM 16bit (which is the only audio format that android soundcard supports) and write it on AudioTrack.

totten
  • 2,769
  • 3
  • 27
  • 41
4

Apologies if the following is dumb:

The ffmpeg command line appears to be generating a test sound and emitting it as a pcm data stream over RTP.

RTP in itself does not guarantee reliable delivery of streamed data, it merely provides enough information to tell the receiver if it has received all the data, and exactly what data is missing if some was lost in transit. Plus it is normally used over UDP.

Hence with RTP the emphasis is on the user of RTP to send data that is encoded in such a way (i.e. with error correction coding, redundancy in the data, etc) so that the reciever can reconstruct enough of the original data to meet the application's needs. So with an audio stream you'd need some sort of encoding format that suits.

I've not found a reference for what pcm_u8 means but it is highly suggestive of it being a straightforward pulse code modulated data stream, with 8 bit data. That doesn't sound like it has any error correction encoding or data redundancy built into it. Losing a byte of that means losing a sample, and there's nothing that can be done at the receiving end to fill in.

So I think what's happening is that something in your network is dropping UDP packets, that RTP is telling the AudioStream which data is missing and the result is gaps because there's no error correction or data redundancy in the pcm_u8 data stream to allow the lost data to be reconstructed by the AudioStream.

I've seen things like VMWare deliberately drop UDP packets on a virtual network as a way of ensuring good performance, the justification being that UDP is not guaranteed delivery anyway so "it doesn't matter". That severely stung a colleague who was using RTP and was expecting guaranteed delivery but didn't get it. He had a closed network segment with a sever at each end of the wire, one of them hosting a single VM.

So it might simply be a case of changing which codec you're using. I'm not able to recommend one. For a start it's worth examining what a broadcast digital media stream uses. DVB-T uses MPEG Transport Stream (which has error correction coding, etc) as, AFAIK, a wrapper around MPEG-2.

bazza
  • 7,580
  • 15
  • 22