Have you tried JXTrayIcon?
I tested this demo from SwingHelper on Ubuntu 10.10 with Compiz and it looks cool.
UPDATE
As 2020, these links are broken and this solution has many drawbacks today. For instance, GNOME3 desktop environments had removed entirely system tray icons and they replaced it with AppIndicator.
Java's (AWT/Swing) System Tray support is broken today. I recommend using this Java library: https://github.com/dorkbox/SystemTray
From the site:
Professional, cross-platform SystemTray support for Swing/AWT, GtkStatusIcon, and AppIndicator on Java 6+. This library provides OS Native menus and Swing/AWT menus, depending on the OS and Desktop Environment and if AutoDetect (the default) is enabled.
For reference, you can found a copy of the original example at here