SunPKCS11 provider in MacOS for Firefox

Question

I managed to get the SunPKCS11 to work with Firefox ESR 52.0 under Windows, but I am unable to get it to load in MacOS. I've already tried several different configuration and directly loading it via PKCS11 but nothing worked, can anyone give me some pointer?

pkcs11.cfg configuration as follow:

name = FirefoxKeyStore
library = "/Applications/Firefox.app/Contents/MacOS/fixed-for-java-runtime/libsoftokn3.dylib"
attributes = compatibility
nssArgs = "configdir='/Users/helloworld/Library/Application Support/Firefox/Profiles/wasdwasd.default-1453211557245' certPrefix='' keyPrefix='' secmod='secmod.db' flags='readOnly' "
slot = 2

Then in Java, I've tried to load it like this:

FileInputStream fis = new FileInputStream("pkcs11.cfg");
Provider provider = new SunPKCS11(fis);
Security.addProvider(provider);

However this immediately gives me the following error:

sunpkcs11: Initializing PKCS#11 library /Applications/Firefox.app/Contents/MacOS/fixed-for-java-runtime/libsoftokn3.dylib
sunpkcs11: Multi-threaded initialization failed: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_DEVICE_ERROR
Exception in thread "main" java.security.ProviderException: Initialization failed
at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:376)
at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:107)

You maybe asking why do I load the .dylib from the weird folder, that's because I used install_name_tool in MacOS to change the @executable_path to @loader_path in order to get the library dependency to work (since I'm trying to run this in Eclipse rather than from Firefox itself).

I also tried to use the solution suggested here: How to finalize SunPKCS11 Provider after it is initialized? and it was a no-go either... I get the same error.

As well as trying various different configuration settings mentioned here: https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/PKCS11/Module_Specs#Softoken_Specific_Parameters

EDIT1

I tried the method mentioned by @FaithReaper but it still throw the same error. I tried changing the slot value to 0, 1 or -1 and result the same. It looks like there are problem loading the underlaying PKCS11 object.

Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_DEVICE_ERROR
at sun.security.pkcs11.wrapper.PKCS11.C_Initialize(Native Method)
at sun.security.pkcs11.wrapper.PKCS11$SynchronizedPKCS11.C_Initialize(PKCS11.java:1545)
at sun.security.pkcs11.wrapper.PKCS11.getInstance(PKCS11.java:157)
at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:330)

I dunno if this would help, but I ran the modutil on the FireFox profile and is dumping this info:

modutil -dbdir "/Users/eto/Library/Application Support/Firefox/Profiles/ew2g332o.default-1453211557245" -rawlist

library= name="NSS Internal PKCS #11 Module" 
parameters="configdir=/Users/eto/Library/Application Support/Firefox/Profiles/ew2g332o.default-1453211557245 certPrefix= keyPrefix= secmod=secmod.db flags=readOnly " 
NSS="Flags=internal,critical trustOrder=75 cipherOrder=100 slotParams=(1={slotFlags=[RSA,DSA,DH,RC2,RC4,DES,RANDOM,SHA1,MD5,MD2,SSL,TLS,AES,Camellia,SEED,SHA256,SHA512] askpw=any timeout=30})"

Listing of PKCS #11 Modules

1. NSS Internal PKCS #11 Module uri: pkcs11:library-manufacturer=Mozilla%20Foundation;library-description=NSS%20Internal%20Crypto%20Services;library-version=3.33 slots: 2 slots attached status: loaded

   slot: NSS Internal Cryptographic Services token: NSS Generic Crypto Services uri: pkcs11:token=NSS%20Generic%20Crypto%20Services;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203

   slot: NSS User Private Key and Certificate Services token: NSS Certificate DB uri: pkcs11:token=NSS%20Certificate%20DB;manufacturer=Mozilla%20Foundation;serial=0000000000000000;model=NSS%203

Answer 1

First of all, I have noticed the different way you insert/add Provider with me. Can you try to add the Provider this way? (it may be irrelavent)

Provider p = new SunPKCS11(new ByteArrayInputStream(config.getBytes()));
Security.insertProviderAt(p, 1);
KeyStore.Builder builder = null;
builder = KeyStore.Builder.newInstance("PKCS11", p, 
    new KeyStore.CallbackHandlerProtection(new UtilTarjetas().new CustomCallbackHandler()));
cardKeyStore = builder.getKeyStore();

Then, maybe you can try the approach here:

https://github.com/avocado-framework/avocado/issues/1112

That is:

apahim commented on 7 Apr 2016

@will-Do, nss folks were able to track this issue down and seems like they are going to consider a change in NSS_InitContext(), which should include a SECMOD_RestartModules(PR_FALSE) after a fork(). Anyway, they also provided a better workaround than the one I provided you. If you set the environment variable NSS_STRICT_NOFORK to DISABLED then the code is expected to work. It worked for me and I'd like to check if it works for you as well. Looking forward to see your results.

Also, some sources suggests that it may be a token or slot problem. You can try to change slot index to 0, or -1.

Answer 2

The libraries provided by Firefox in Mac OS X might actually be completely broken or non-functional if execute outside of FireFox.

After trying many different combination of configuration and ways, I finally got it working by NOT using the libraries from Firefox...

Here is the step by step ways to get it to work:

1. Install homebrew in your Mac
2. Run brew install nss
3. Run brew install nspr
4. You may need to manually link them with brew link nss and brew link nspr
5. In the configuration file, you need to point it to your homebrew libsoftokn3.dylib, like so library = /usr/local/opt/nss/lib/libsoftokn3.dylib

Then your Java code should be able to load the Firefox keystore under Mac OS X... I've filed a bug here. The ticket contains a working sample code on how to instantiate the PKCS11, load the Firefox keystore and list the aliases from the store.

This was definitely a nightmare working with Firefox... but at least I got it working... Who would've thought the libraries they provided doesn't work (but it worked in Windows!)? :P

For completeness, I've included the sample code directly in this post as well:

import java.io.ByteArrayInputStream;
import java.security.KeyStore;
import java.security.Provider;
import java.security.Security;
import java.util.Collections;

public class Sample {

    private KeyStore load(String lib, String profile) throws Exception
    {
        String config = "library = " + lib + "\n" + 
                "name = FirefoxKeyStore\n" + 
                "attributes = compatibility\n" + 
                "nssArgs = \"configDir='" + profile + "' certPrefix='' keyPrefix='' secmod='secmod.db' flags='readOnly,forceOpen,optimizeSpace' \"\n" + 
                "slot = 2\n";

        ByteArrayInputStream bais = new ByteArrayInputStream(config.getBytes());
        Provider provider = new sun.security.pkcs11.SunPKCS11(bais);
        Security.addProvider(provider);

        return KeyStore.getInstance("PKCS11");
    }

    public static void main(String[] args) throws Exception {

        Sample s = new Sample();

        String profile = "/Users/blah/Library/Application Support/Firefox/Profiles/yougottachangethis";
        String[] libs = { 
            //"/Applications/Firefox.app/Contents/MacOS/libsoftokn3.dylib",
            "/usr/local/opt/nss/lib/libsoftokn3.dylib"
        };

        for (String lib : libs) {
            System.out.println("TRYING >>> " + lib);
            try {
                KeyStore ks1 = s.load(lib, profile);

                ks1.load(null, null);
                for (String alias : Collections.list(ks1.aliases())) {
                    System.out.println(alias);  
                }
            }
            catch (Exception e)
            {
                e.printStackTrace();
            }
        }
    }
}

Here is the output of the otool from Mac (included for FaithReaper's comments):

otool -L libsoftokn3.dylib
libsoftokn3.dylib:
/usr/local/opt/nss/lib/libsoftokn3.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libsqlite3.dylib (compatibility version 9.0.0, current version 253.0.0)
/usr/local/Cellar/nss/3.34/lib/libnssutil3.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/local/opt/nspr/lib/libplc4.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/local/opt/nspr/lib/libplds4.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/local/opt/nspr/lib/libnspr4.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)