You need to set the look and feel. The default look and feel for java is called metal, and that is what you are seeing in the first image. The second one appears to be windows, which is the system look and feel.
Another cleaner looking option is nimbus. When you are more advanced you can create your own or install new look and feels someone else has made.
http://docs.oracle.com/javase/tutorial/uiswing/lookandfeel/plaf.html
Use this code to set the system look and feel:
try {
// Set System L&F
UIManager.setLookAndFeel(
UIManager.getSystemLookAndFeelClassName());
}
catch (UnsupportedLookAndFeelException e) {
// handle exception
}
catch (ClassNotFoundException e) {
// handle exception
}
catch (InstantiationException e) {
// handle exception
}
catch (IllegalAccessException e) {
// handle exception
}
Here is the link for the nimbus look and feel:
http://docs.oracle.com/javase/tutorial/uiswing/lookandfeel/nimbus.html
Make sure you set the look and feel in the very beginning of your program, before you create and gui components.