Java ConcurrentModificationException thrown while inside a synchronized block

Question

I am working on a pretty basic server that accepts clients, and will write out data from one client to all of the other clients connected. Right now it works fine until one of the clients disconnects, at which point the next time the server attempts to iterate over the clients list, I get a concurentmodification exception out of the handle method. The way I understood the synchronized(clients) keyword was that once a thread called handle and began iterating over the client list, any thread that attempted to call add/remove on that list would block until handle was finished.

private void addClientThread(Socket _socket)
{
    System.out.println("[DEBUG]: Adding client: "+_socket.getPort());
    serverThread temp = new serverThread(this,_socket);

    try
    {
        System.out.println("Server accepted new client: " + _socket+".  Spawning new handler thread");
        temp.open();
        synchronized(clients)
        {
            clients.add(temp);
        }
        temp.start();
    }
    catch(IOException ioe)
    {
        System.out.println("Error opening thread: " + ioe);
    }
}

public void remove(serverThread threadToRemove)
{
    synchronized(clients)
    {
        clients.remove(threadToRemove);
    }
    try
    {
        threadToRemove.close();
        System.out.println("Server closed client handler thread: "+threadToRemove.getId());
    }
    catch(IOException ex)
    {
        System.out.println("IO Exception while closing thread: "+ex);
    }
}

public void handle(serverThread self,byte input)
{
    synchronized(clients)
    {
        // Pass anything the client wrote to all of the other clients
        for(serverThread temp : clients)
        {
            if(!temp.equals(self))
                temp.send(input);
        }
    }
}