Adding MouseListener breaks mouse event dispatching

Question

I'm currently trying to fix a bug that was introduced by adding an arbitrary java.awt.event.MouseListener to a java.awt.Component. The following code is a reduced version of a drag and drop implementation for components:

1. When a component gets pressed it is removed from its parent and can be dragged around the screen in a second window
2. When the user releases the mouse, the component is added to some other component (for simplicity it gets added to its original parent again in the provided example)

Here is the code:

import java.awt.AWTEvent;
import java.awt.Color;
import java.awt.Toolkit;
import java.awt.event.AWTEventListener;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;

import javax.swing.BorderFactory;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.JWindow;
import javax.swing.SwingUtilities;

public class DragComponentBug {


    public static void main(String[] args) {
        // set to true to enable the broken behavior
        boolean addMouseListener = false;

        // set to false to fix the broken behavior by not removing the label from its parent
        boolean removeLabel = true;

        SwingUtilities.invokeLater(new Runnable() {

            @Override
            public void run() {
                JFrame frame = new JFrame();
                frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
                frame.setLocation(100, 100);
                JWindow dragWindow = new JWindow();
                if (!removeLabel) {
                    JPanel panel = new JPanel();
                    panel.setBorder(BorderFactory.createLineBorder(Color.black));
                    dragWindow.add(panel);
                }
                JLabel label = new JLabel("Drag me around");
                frame.add(label);
                MouseAdapter labelListener = new MouseAdapter(){
                    @Override
                    public void mousePressed(MouseEvent e) {
                        System.err.println("mouse pressed: " + e);
                    }

                    @Override
                    public void mouseReleased(MouseEvent e) {
                        System.err.println("mouse released: " + e);
                    }

                    @Override
                    public void mouseDragged(MouseEvent e) {
                        System.err.println("mouse dragged: " + e);
                    }
                };
                if (addMouseListener) {
                    label.addMouseListener(labelListener);
                    label.addMouseMotionListener(labelListener);
                }

                Toolkit.getDefaultToolkit().addAWTEventListener(new AWTEventListener() {
                    @Override
                    public void eventDispatched(AWTEvent event) {
                        MouseEvent e = (MouseEvent)event;
                        switch (e.getID()) {
                        case MouseEvent.MOUSE_PRESSED:
//                          System.out.println("mouse pressed:" + event);
                            if (removeLabel) {
                                label.setBorder(BorderFactory.createLineBorder(Color.black));
                                dragWindow.add(label);
                            }
                            dragWindow.setLocation(e.getLocationOnScreen());
                            dragWindow.setSize(label.getSize());
                            dragWindow.setVisible(true);
                            frame.repaint();
                            break;
                        case MouseEvent.MOUSE_RELEASED:
//                          System.out.println("mouse released:" + event);
                            if (removeLabel) {
                                label.setBorder(null); 
                                frame.add(label);
                            }
                            frame.repaint();
                            dragWindow.setVisible(false);
                            break;
                        case MouseEvent.MOUSE_DRAGGED:
//                          System.out.println("mouse dragged:" + event);
                            dragWindow.setLocation(e.getLocationOnScreen());
                            break;
                        default:
                            break;
                        }
                    }
                }, AWTEvent.MOUSE_EVENT_MASK | AWTEvent.MOUSE_MOTION_EVENT_MASK);
                frame.pack();
                frame.setVisible(true);
                frame.addWindowListener(new WindowAdapter() {
                    @Override
                    public void windowClosed(WindowEvent e) {
                        dragWindow.dispose();
                    }
                });
            }
        });
    }

}

Now for some reason, if one adds a MouseListener (or also a MouseMotionListener) of any kind to the dragged label, then the drag and drop behavior is broken and the global AWTEventListener does not receive any dragging or releasing mouse events (neither does the added listener). This can be tested by setting addMouseListener to true in the provided example.

Another important aspect here is, that this behavior somehow seems to be related to the fact, that the label gets removed from its parent after the mouse was pressed. If one does not remove the label, then the event dispatching will continue to work. This can be tested by setting removeLabel to false after addMouseListener was set to true.

Why are no more mouse events dispatched after the listener gets added to the label and how does this relate to removing the label from its parent? Is there any way to restore the intended behavior even if a listener is added to the label (including removing/adding the label during the drag operation)?

Thanks for any help