So, I did a really quick test using...
import java.awt.EventQueue;
import javax.swing.JFrame;
import javax.swing.JPanel;
import java.io.IOException;
import javax.imageio.ImageIO;
import javax.swing.JLabel;
import com.apple.eawt.Application;
public class Test {
public static void main(String[] args) {
try {
Application application = Application.getApplication();
application.setDockIconImage(ImageIO.read(Test.class.getResource("/javaapplication163/bunny.jpg")));
} catch (IOException ex) {
ex.printStackTrace();
}
new Test();
}
public Test() {
EventQueue.invokeLater(new Runnable() {
@Override
public void run() {
JFrame frame = new JFrame();
frame.add(new TestPane());
frame.pack();
frame.setLocationRelativeTo(null);
frame.setVisible(true);
}
});
}
public static class TestPane extends JPanel {
public TestPane() {
add(new JLabel("Hello"));
}
}
}
And it showed...

Neat ... however, this will only compile and run on a Mac.
A "typical" solution to this would be to use reflection to try and load the com.apple.eawt.Application
at run time, something like...
try {
Class appClass = Class.forName("com.apple.eawt.Application");
Class params[] = new Class[]{Image.class};
Method getApplication = appClass.getMethod("getApplication");
Object application = getApplication.invoke(appClass);
Method setDockIconImage = appClass.getMethod("setDockIconImage", params);
setDockIconImage.invoke(application, ImageIO.read(Test.class.getResource("/javaapplication163/bunny.jpg")));
} catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException | ClassNotFoundException | IOException ex) {
ex.printStackTrace();
}
But this is some what error prone and messy, not to mention a complete pain in the ... code to maintain.
So, then I thought I might try using the command line parameters, as demonstrated at The dock icon...
-Xdock:name="Hello bunny" -Xdock:icon=/path/to/bunny.jpg
which also worked.
While I kind of prefer this solution, it does mean that the icon needs to be externalised from Jar, which is kind of a pain, and you'll probably need to supply a full working path.
If I was to generate a App bundle for my project, I'd probably follow this, but if I was just deploying a Jar, I might be tempted to use one of the previous methods