Java Voice Chat - Mixdown Incoming Data for Single Output

Question

I am trying to write a Java voice chat application, and have achieved echo capabilities, but when trying to connect multiple clients, I am stuck. I understand that you cannot iterate through sockets and send the data to everyone connected without mixing-down the data. (I have tried and it sounds nothing like how it should). I am not quite sure what to do, and I am using a very simple byte-buffered echo server as the server (where I would like to perform the mixdown). I also have a client that takes microphone input, sends it to the server, takes data from the server, and plays that data out of a speaker.

NOTE: The client is composed of 2 classes (Program and SoundReceiver). I am using the javax.sound.sampled library.

Echo Server: http://pastebin.com/c9KiaTpJ

import java.net.*;
import java.io.*;
import java.util.*;

public class Echo
{
    public static void main(String[] args) throws Exception
    {
        ServerSocket serverSocket = new ServerSocket(3000);
        while(true){Thread echoThread = new Thread(new EchoThread(serverSocket.accept()));
                    echoThread.start();}
    }
}

class EchoThread implements Runnable
{
    public static Collection<Socket> sockets = new ArrayList<Socket>();
    Socket connection = null;
    DataInputStream dataIn = null;
    DataOutputStream dataOut = null;

    public EchoThread(Socket conn) throws Exception
    {
        connection = conn;
        dataIn = new DataInputStream(connection.getInputStream());
        dataOut = new DataOutputStream(connection.getOutputStream());
        sockets.add(connection);
    }

    public void run()
    {
        int bytesRead = 0;
        byte[] inBytes = new byte[1];
        while(bytesRead != -1)
        {
            try{bytesRead = dataIn.read(inBytes, 0, inBytes.length);}catch (IOException e){}
            if(bytesRead >= 0)
            {
                sendToAll(inBytes, bytesRead);
            }
        }
        sockets.remove(connection);
    }

    public static void sendToAll(byte[] byteArray, int q)
    {
        Iterator<Socket> sockIt = sockets.iterator();
        while(sockIt.hasNext())
        {
            Socket temp = sockIt.next();
            DataOutputStream tempOut = null;
            try
            {
                tempOut = new DataOutputStream(temp.getOutputStream());
            } catch (IOException e1)
            {
                // TODO Auto-generated catch block
                e1.printStackTrace();
            }
            try{tempOut.write(byteArray, 0, q);}catch (IOException e){}
        }
    }
}

Client Program Class: http://pastebin.com/v24CYwXE

import java.io.DataOutputStream;
import java.net.*;
import javax.sound.sampled.*;

public class Program
{
    public static void main(String[] args) throws Exception
    {
        AudioFormat af = new AudioFormat(8000.0f,8,1,true,false);
        DataLine.Info info = new DataLine.Info(TargetDataLine.class, af);
        TargetDataLine microphone = (TargetDataLine)AudioSystem.getLine(info);
        microphone.open(af);
        Socket conn = new Socket("localhost",3000);
        microphone.start();
        DataOutputStream dos = new DataOutputStream(conn.getOutputStream());
        int bytesRead = 0;
        byte[] soundData = new byte[1];
        Thread inThread = new Thread(new SoundReceiver(conn));
        inThread.start();
        while(bytesRead != -1)
        {
            bytesRead = microphone.read(soundData, 0, soundData.length);
            if(bytesRead >= 0)
            {
                dos.write(soundData, 0, bytesRead);
            }
        }
        System.out.println("IT IS DONE.");
    }
}

Client SoundReceiver Class: http://pastebin.com/2tt0Jucv

import java.net.*;
import java.io.*;

import javax.sound.sampled.*;

public class SoundReceiver implements Runnable
{
    Socket connection = null;
    DataInputStream soundIn = null;
    SourceDataLine inSpeaker = null;

    public SoundReceiver(Socket conn) throws Exception
    {
        connection = conn;
        soundIn = new DataInputStream(connection.getInputStream());
        AudioFormat af = new AudioFormat(8000.0f,8,1,true,false);
        DataLine.Info info = new DataLine.Info(SourceDataLine.class, af);
        inSpeaker = (SourceDataLine)AudioSystem.getLine(info);
        inSpeaker.open(af);
    }

    public void run()
    {
        int bytesRead = 0;
        byte[] inSound = new byte[1];
        inSpeaker.start();
        while(bytesRead != -1)
        {
            try{bytesRead = soundIn.read(inSound, 0, inSound.length);} catch (Exception e){}
            if(bytesRead >= 0)
            {
                inSpeaker.write(inSound, 0, bytesRead);
            }
        }
    }
}

Basically, I'd like to merge all incoming bytes into one byte array while keeping everyone's voice full (just like a 3-way phone call).

Personally (and I know I'm not the only one), I tend to be too lazy to click on links like that. It's generally a good idea to inline your code in your question. And if you feel that inlining all of your code would be too much, creating a [SSCCE](http://sscce.org), if possible, is always a good idea. — Jeffrey, Jul 24 '12 at 02:29
Is there an easier way to post code than making 4 spaces before every line? — zgillis, Jul 24 '12 at 02:50
Looks like this has not much to do with sockets, but everything to do with sound processing. — Nikolai Fetissov, Jul 24 '12 at 03:01
Yes, I really just need to merge the data and have it sound right. Beyond that, the DataOutputStreams and DataInputStreams will be used. — zgillis, Jul 24 '12 at 03:10

score 1 · Answer 1 · answered Jun 11 '13 at 06:58

1

setting the limit to the serverSocket may help, eg new ServerSocket(3000,101); something like backlog or queue length..

answered Jun 11 '13 at 06:58

kAmol

1,297
1
18
29

score 1 · Answer 2 · edited Jan 15 '14 at 13:59

1

It's this line:

try
{
    tempOut.write(byteArray, 0, q);
}
catch (IOException e){
}

in server side which i think sends the data back to the client due which there is an echo. I think you should omit the line.

edited Jan 15 '14 at 13:59

Markus

20,838
4
31
55

answered Jan 15 '14 at 13:40

sam

83
1
7

score 0 · Answer 3 · answered Jan 17 '14 at 05:16

I think you need to create a check in the server side. its like if sendAll is being called from EChoThread that has connection instance just pass this to sendAll and there compare sockIt with connection if they are the same then this is the same socket that is sending the date and there is no need to send data to it self so just skip it and move to the next socket.

the following changes should be made at server side:

     public void run()
{
    int bytesRead = 0;
    byte[] inBytes = new byte[1];
    while(bytesRead != -1)
    {
        try{bytesRead = dataIn.read(inBytes, 0, inBytes.length);}catch (IOException e)       {}
        if(bytesRead >= 0)
        {
            sendToAll(connection, inBytes, bytesRead);
        }
    }
    sockets.remove(connection);
}


public static void sendToAll(Socket connection, byte[] byteArray, int q)
{
    Iterator<socket> sockIt = sockets.iterator();
    while(sockIt.hasNext())
    {
        Socket temp = sockIt.next();
        if(connection == temp){
            continue;
        }
        DataOutputStream tempOut = null;
        try
        {
            tempOut = new DataOutputStream(temp.getOutputStream());
        } catch (IOException e1)
        {
            // TODO Auto-generated catch block
            e1.printStackTrace();
        }
        try{tempOut.write(byteArray, 0, q);}catch (IOException e){}
      }
  }




 public void run()
{
    int bytesRead = 0;
    byte[] inBytes = new byte[1];
    while(bytesRead != -1)
    {
        try{bytesRead = dataIn.read(inBytes, 0, inBytes.length);}catch (IOException e)       {}
        if(bytesRead >= 0)
        {
            sendToAll(connection, inBytes, bytesRead);
        }
    }
    sockets.remove(connection);
}


public static void sendToAll(Socket connection, byte[] byteArray, int q)
{
    Iterator<socket> sockIt = sockets.iterator();
    while(sockIt.hasNext())
    {
        Socket temp = sockIt.next();
        if(connection == temp){
            continue;
        }
        DataOutputStream tempOut = null;
        try
        {
            tempOut = new DataOutputStream(temp.getOutputStream());
        } catch (IOException e1)
        {
            // TODO Auto-generated catch block
            e1.printStackTrace();
        }
        try{tempOut.write(byteArray, 0, q);}catch (IOException e){}
      }
  }

Java Voice Chat - Mixdown Incoming Data for Single Output

3 Answers3