Dragging over JButtons/JLabels

Question

I'm trying to make the game Dots and I have already figured out a way to generate random dots using an int[][]. I've also created a method to remove dots if they are next to each other and have the same colour when selected. My playfield is 7 by 7 and is filled with JButtons (which could also be JLabels). Now I need to find a way to drag over a few dots and select them if they are the same colour as the initial dot, but whenever I try to do something like that, I always end up either selecting the initial dot itself and send that button's coordinates whenever I move the mouse, or select the x and y values for the pointer on the screen.

Here is the relevant code:

for (int i = 0; i < 7; i++) {
    for (int j = 0; j < 7; j++) {
        final int finalJ = j;
        final int finalI = i;
        playField[i][j].addMouseMotionListener(new MouseMotionAdapter() {
            @Override
            public void mouseDragged(MouseEvent e) {
                try {
                    while (true) {
                        dotsController.beginZet(finalI, finalJ);
                    }
                } catch (DotsException e1) {
                    e1.printStackTrace();
                }
            }
        });
        playField[i][j].addMouseListener(new MouseAdapter() {
            @Override
            public void mouseReleased(MouseEvent e) {
                try {
                    dotsController.doeZet();
                } catch (DotsException e1) {
                    e1.printStackTrace();
                }
            }
        });
    }
}