This depends a lot on the OS you're targeting. If you're trying to go for Windwos, for example, you might check out this article, which describes JDIC, and has sample code (which I've included below) that should help you do this.
http://java.sun.com/developer/technicalArticles/J2SE/Desktop/jdic_assoc/
Create a file association:
import org.jdesktop.jdic.desktop.*;
import org.jdesktop.jdic.filetypes.*;
AssociationService serv = new AssociationService();
Association logassoc = new Association();
// Adds the .log type to the Association object.
logassoc.addFileExtension("LOG");
// Adds an Action to the Association object that will
// open a .log file with Windows Notepad.
logassoc.addAction(
new org.jdesktop.jdic.filetypes.Action(
"open",
"C:\\WINDOWS\\system32\\NOTEPAD.EXE %1"));
try {
// Adds the .log Association to the file types' table
// at the user level using an AssociationService object.
serv.registerUserAssociation(logassoc);
}
catch (java.lang.IllegalArgumentException e) {
// This exception will be caught if the given Association is not valid
// to be added to the table of file types.
System.err.println(e);
}
catch (AssociationAlreadyRegisteredException e) {
// This exception will be caught if the Association already
// exists in the table of file types.
System.err.println(e);
}
catch (RegisterFailedException e) {
// This exception will be caught if the Association was
// unable to be added to the table of file types.
System.err.println(e);
}
Remove an association:
import org.jdesktop.jdic.desktop.*;
import org.jdesktop.jdic.filetypes.*;
AssociationService serv = new AssociationService();
// This uses an AssociationService to search the table of file
// types for the .log extension. If the .log file is found,
// an Association object representing the .log file type
// will be returned. Otherwise, null is returned.
Association logassoc = serv.getFileExtensionAssociation("LOG");
try {
// The AssociationService will remove the .log file type from
// the table of file types.
serv.unregisterUserAssociation(logassoc);
} catch (java.lang.IllegalArgumentException e) {
// This exception will be caught if the given Association is not valid
// to be removed from the table of file types.
System.err.println(e);
} catch (AssociationNotRegisteredException e) {
// This exception will be caught if the Association does not already
// exist in the table of file types.
System.err.println(e);
} catch (RegisterFailedException e) {
// This exception will be caughtif the association was unable to be
// removed from the table of file types.
System.err.println(e);
}