Problems With MouseListener

Question

I'm writing a game in java. The problem here is I wrote my game to run in a JFrame, not thinking that i would want to add menus and a results screen and all that good stuff. The game itself runs great in the JFrame. What i decided to do, though, was turn my JFrame into a JPanel, create a separate class for my JFrame and then just add my JPanel to the frame. Everything works just peachy except my MouseListener no longer does a darn thing. Can someone tell me how to make this work or a different idea of how this can be done?

/////UPDATE So apparently i found an answer while recreating the problem.... I just need the figure out the difference between my game code and the test code.

Here is the example i wrote up to try and reproduce the problem.. Oddly enough this works. Now I'm even more confused. So apparently this is ok:

//Class for the JFrame
package mousetest;

import java.awt.Color;
import javax.swing.JFrame;

public class MouseTest extends JFrame{

public static void main(String[] args) {
    MouseTest test = new MouseTest();
}

public MouseTest(){
    //create teh board
    Board game = new Board();

    //framestuff
    setSize(406, 630);
    setLocationRelativeTo(null);
    setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    setResizable(false);
    setVisible(true);
    setBackground(Color.black);
    add(game); // add it
}

}

========================================================================

//Class for the JPanel that my game is in
package mousetest;

import java.awt.Color;
import java.awt.event.MouseEvent;
import java.awt.event.MouseListener;
import javax.swing.JLabel;
import javax.swing.JPanel;

public class Board extends JPanel{
    JLabel testlabel = new JLabel("testtext");
    //CONSTRUCTOR
    public Board(){
        setBackground(Color.WHITE);
        testlabel.addMouseListener(new Mousehandle());
        setVisible(true);
        add(testlabel);
    }


    // control ALLTHECLICKS!!!!!
    class Mousehandle implements MouseListener{

        public Mousehandle(){

        }

        public void mouseClicked(MouseEvent e) {

        }

        public void mousePressed(MouseEvent e) {
            if(e.getSource() == testlabel){
                System.out.println("mouse down");
            }
        }

        public void mouseReleased(MouseEvent e) {
            if(e.getSource() == testlabel){
                System.out.println("mouse up");
            }
        }

        public void mouseEntered(MouseEvent e) {
            if(e.getSource() == testlabel){
                System.out.println("rollover");
            }

        }

        public void mouseExited(MouseEvent e) {
           if(e.getSource() == testlabel){
                System.out.println("roll off");
            }
        }

        public void mouseDragged(MouseEvent e){

        }

    }


}