Getting BouncyCastle to decrypt a GPG-encrypted message

Question

How can I get BouncyCastle to decrypt a GPG-encrypted message?

I have created a GPG key pair at the CentOS 7 command line using gpg --gen-key. I chose RSA RSA as the encryption types, and I exported the keys using gpg --export-secret-key -a "User Name" > /home/username/username_private.key and gpg --armor --export 66677FC6 > /home/username/username_pubkey.asc

I am able to import username_pubkey.asc into a remote Thunderbird client of another email account and successfully send an encrypted email to username@mydomain.com. But when my Java/BouncyCastle code running at mydomain.com tries to decrypt the GPG-encoded data, it gives the following error:

org.bouncycastle.openpgp.PGPException:  
Encrypted message contains a signed message - not literal data.

If you look at the code below, you will see this corresponds with the line in PGPUtils.decryptFile() which states else if (message instanceof PGPOnePassSignatureList) {throw new PGPException("Encrypted message contains a signed message - not literal data.");

The original code for this came from the blog entry at this link, though I made minor changes to get it to compile in Eclipse Luna with Java 7. A user of the linked blog reported the same error, and the blog author replied by saying that it does not work with GPG. So how do I fix this to make it work with GPG?

The Java decryption code starts when the GPG-encoded-file and the GPG-secret-key are passed into Tester.testDecrypt() as follows:

Tester.java contains:

public InputStream testDecrypt(String input, String output, String passphrase, String skeyfile) throws Exception {
    PGPFileProcessor p = new PGPFileProcessor();
    p.setInputFileName(input);//this is GPG-encoded data sent from another email address using Thunderbird
    p.setOutputFileName(output);
    p.setPassphrase(passphrase);
    p.setSecretKeyFileName(skeyfile);//this is the GPG-generated key
    return p.decrypt();//this line throws the error
}

PGPFileProcessor.java includes:

public InputStream decrypt() throws Exception {
    FileInputStream in = new FileInputStream(inputFileName);
    FileInputStream keyIn = new FileInputStream(secretKeyFileName);
    FileOutputStream out = new FileOutputStream(outputFileName);
    PGPUtils.decryptFile(in, out, keyIn, passphrase.toCharArray());//error thrown here
    in.close();
    out.close();
    keyIn.close();
    InputStream result = new FileInputStream(outputFileName);//I changed return type from boolean on 1/27/15
    Files.deleteIfExists(Paths.get(outputFileName));//I also added this to accommodate change of return type on 1/27/15
    return result;
}

PGPUtils.java includes:

/**
 * decrypt the passed in message stream
 */
@SuppressWarnings("unchecked")
public static void decryptFile(InputStream in, OutputStream out, InputStream keyIn, char[] passwd)
    throws Exception
{
    Security.addProvider(new BouncyCastleProvider());

    in = org.bouncycastle.openpgp.PGPUtil.getDecoderStream(in);

    //1/26/15 added Jca prefix to avoid eclipse warning, also used https://www.bouncycastle.org/docs/pgdocs1.5on/index.html
    PGPObjectFactory pgpF = new JcaPGPObjectFactory(in);
    PGPEncryptedDataList enc;

    Object o = pgpF.nextObject();
    //
    // the first object might be a PGP marker packet.
    //
    if (o instanceof  PGPEncryptedDataList) {enc = (PGPEncryptedDataList) o;}
    else {enc = (PGPEncryptedDataList) pgpF.nextObject();}

    //
    // find the secret key
    //
    Iterator<PGPPublicKeyEncryptedData> it = enc.getEncryptedDataObjects();
    PGPPrivateKey sKey = null;
    PGPPublicKeyEncryptedData pbe = null;

    while (sKey == null && it.hasNext()) {
        pbe = it.next(); 
        sKey = findPrivateKey(keyIn, pbe.getKeyID(), passwd);
    }

    if (sKey == null) {throw new IllegalArgumentException("Secret key for message not found.");}

    InputStream clear = pbe.getDataStream(new BcPublicKeyDataDecryptorFactory(sKey));

    //1/26/15 added Jca prefix to avoid eclipse warning, also used https://www.bouncycastle.org/docs/pgdocs1.5on/index.html
    PGPObjectFactory plainFact = new JcaPGPObjectFactory(clear);

    Object message = plainFact.nextObject();

    if (message instanceof  PGPCompressedData) {
        PGPCompressedData cData = (PGPCompressedData) message;
        //1/26/15 added Jca prefix to avoid eclipse warning, also used https://www.bouncycastle.org/docs/pgdocs1.5on/index.html
        PGPObjectFactory pgpFact = new JcaPGPObjectFactory(cData.getDataStream()); 
        message = pgpFact.nextObject();
    }

    if (message instanceof  PGPLiteralData) {
        PGPLiteralData ld = (PGPLiteralData) message;

        InputStream unc = ld.getInputStream();
        int ch;

        while ((ch = unc.read()) >= 0) {out.write(ch);}
    } else if (message instanceof  PGPOnePassSignatureList) {
        throw new PGPException("Encrypted message contains a signed message - not literal data.");
    } else {
        throw new PGPException("Message is not a simple encrypted file - type unknown.");
    }

    if (pbe.isIntegrityProtected()) {
        if (!pbe.verify()) {throw new PGPException("Message failed integrity check");}
    }
}

/**
 * Load a secret key ring collection from keyIn and find the private key corresponding to
 * keyID if it exists.
 *
 * @param keyIn input stream representing a key ring collection.
 * @param keyID keyID we want.
 * @param pass passphrase to decrypt secret key with.
 * @return
 * @throws IOException
 * @throws PGPException
 * @throws NoSuchProviderException
 */
public  static PGPPrivateKey findPrivateKey(InputStream keyIn, long keyID, char[] pass)
    throws IOException, PGPException, NoSuchProviderException
{
    //1/26/15 added Jca prefix to avoid eclipse warning, also used https://www.bouncycastle.org/docs/pgdocs1.5on/index.html
    PGPSecretKeyRingCollection pgpSec = new JcaPGPSecretKeyRingCollection(PGPUtil.getDecoderStream(keyIn));
    return findPrivateKey(pgpSec.getSecretKey(keyID), pass);

}

/**
 * Load a secret key and find the private key in it
 * @param pgpSecKey The secret key
 * @param pass passphrase to decrypt secret key with
 * @return
 * @throws PGPException
 */
public static PGPPrivateKey findPrivateKey(PGPSecretKey pgpSecKey, char[] pass)
    throws PGPException
{
    if (pgpSecKey == null) return null;

    PBESecretKeyDecryptor decryptor = new BcPBESecretKeyDecryptorBuilder(new BcPGPDigestCalculatorProvider()).build(pass);
    return pgpSecKey.extractPrivateKey(decryptor);
}

The complete code of all three Java files can be found on a file sharing site by clicking on this link.

The complete stack trace for the error can be found by clicking on this link.

For reference, the GUI instructions for encryption by the remote Thunderbird sender are summarized in the following screen shot:

I have read many postings and links about this. In particular, this other SO posting looks similar, but is different. My Keys use RSA RSA, but the other posting does not.

EDIT#1

As per @DavidHook's suggestion, I have read SignedFileProcessor, and I am starting to read the much longer RFC 4880. However, I need actual working code to study in order to understand this. Most people who find this via google searches will also need working code to illustrate the examples.

For reference, the SignedFileProcessor.verifyFile() method recommended by @DavidHook is as follows. How should this be customized to fix the problems in the code above?

private static void verifyFile(InputStream in, InputStream keyIn) throws Exception {
    in = PGPUtil.getDecoderStream(in);
    PGPObjectFactory pgpFact = new PGPObjectFactory(in);
    PGPCompressedData c1 = (PGPCompressedData)pgpFact.nextObject();
    pgpFact = new PGPObjectFactory(c1.getDataStream());
    PGPOnePassSignatureList p1 = (PGPOnePassSignatureList)pgpFact.nextObject();
    PGPOnePassSignature ops = p1.get(0);
    PGPLiteralData p2 = (PGPLiteralData)pgpFact.nextObject();
    InputStream dIn = p2.getInputStream();
    int ch;
    PGPPublicKeyRingCollection  pgpRing = new PGPPublicKeyRingCollection(PGPUtil.getDecoderStream(keyIn));
    PGPPublicKey key = pgpRing.getPublicKey(ops.getKeyID());
    FileOutputStream out = new FileOutputStream(p2.getFileName());
    ops.initVerify(key, "BC");
    while ((ch = dIn.read()) >= 0){
        ops.update((byte)ch);
        out.write(ch);
    }
    out.close();
    PGPSignatureList p3 = (PGPSignatureList)pgpFact.nextObject();
    if (ops.verify(p3.get(0))){System.out.println("signature verified.");}
    else{System.out.println("signature verification failed.");}
}

EDIT#2

The SignedFileProcessor.verifyFile() method recommended by @DavidHook is almost identical to the PGPUtils.verifyFile() method in my code above, except that PGPUtils.verifyFile() makes a copy of extractContentFile and calls PGPOnePassSignature.init() instead of PGPOnePassSignature.initVerify(). This may be due to a version difference. Also, PGPUtils.verifyFile() returns a boolean, while SignedFileProcessor.verifyFile() gives SYSO for the two boolean values and returns void after the SYSO.

If I interpret @JRichardSnape's comments correctly, this means that the verifyFile() method might best be called upstream to confirm the signature of the incoming file using the sender's public key, and then, if the signature on the file is verified, using another method to decrypt the file using the recipient's private key. Is this correct? If so, how do I restructure the code to accomplish this?

Wow - this is a lot more complicated that I thought it would be!!! One thing I can say tentatively - this is not a problem with the gpg end, because you get the same results if you send the mail with a bouncycastle generated key (I know you have struggled to do that c.f.http://tinyurl.com/p55uxd8), but I ***can*** test that and can send a message with a BC (Java) generated key that Enigmail will decrypt, but this code shows the same problem. The issue appears to be that the code finds the signature and key pair, but then the Inputstream appears to be spent. Investigations continue... — J Richard Snape, Feb 11 '15 at 15:53
These kind of questions may be better answered on the bouncy-dev mailing list, so if you don't get an answer here... — Maarten Bodewes, Feb 11 '15 at 17:47
Sure - the code at the end will work well - you need to understand that the `keyIn` in this case is the *public key of the sender* (or strictly, the public key of whichever key was used to sign the email, I think). This is different from `keyIn` in your other code, which is the private (secret) key of the receiver. Note, though, as per my comment to the answer, you'll only get the text bit of your email - you need to recursively extract the attachments and decrypt if that's your ultimate aim... — J Richard Snape, Feb 12 '15 at 12:09
@JRichardSnape I created a chat room and am available in it to discuss this, if you are willing. I should be online and able to see an alert if you enter the room. Here is the link: http://chat.stackoverflow.com/rooms/70813/someroom — CodeMed, Feb 12 '15 at 18:46
@JRichardSnape If you want to write an answer I would be happy to mark it as accepted and +1. You solved this problem when you showed me how to send an un-signed, but encrypted message to the decryption code. You helped me see that the reason I was getting the error was that the message was signed in addition to being encrypted. I think that the signing issue would warrant a separate question. But for the moment, I am exploring Outlook instead. I will also look for 5 of your other postings that deserve +1 on their own independent, separate, merits. — CodeMed, Feb 24 '15 at 18:26

score 6 · Answer 1 · answered Oct 19 '15 at 14:18

if anyone is interested to know how to encrypt and decrypt gpg files using bouncy castle openPGP library, check the below java code:

The below are the 4 methods you going to need:

The below method will read and import your secret key from .asc file:

public static PGPSecretKey readSecretKeyFromCol(InputStream in, long keyId) throws IOException, PGPException {
in = PGPUtil.getDecoderStream(in);
PGPSecretKeyRingCollection pgpSec = new PGPSecretKeyRingCollection(in, new BcKeyFingerprintCalculator());

PGPSecretKey key = pgpSec.getSecretKey(keyId);

if (key == null) {
    throw new IllegalArgumentException("Can't find encryption key in key ring.");
}
return key;
}

The below method will read and import your public key from .asc file:

@SuppressWarnings("rawtypes")
public static PGPPublicKey readPublicKeyFromCol(InputStream in) throws IOException, PGPException {
    in = PGPUtil.getDecoderStream(in);
    PGPPublicKeyRingCollection pgpPub = new PGPPublicKeyRingCollection(in, new BcKeyFingerprintCalculator());
    PGPPublicKey key = null;
    Iterator rIt = pgpPub.getKeyRings();
    while (key == null && rIt.hasNext()) {
        PGPPublicKeyRing kRing = (PGPPublicKeyRing) rIt.next();
        Iterator kIt = kRing.getPublicKeys();
        while (key == null && kIt.hasNext()) {
            PGPPublicKey k = (PGPPublicKey) kIt.next();
            if (k.isEncryptionKey()) {
                key = k;
            }
        }
    }
    if (key == null) {
        throw new IllegalArgumentException("Can't find encryption key in key ring.");
    }
    return key;
}

The below 2 methods to decrypt and encrypt gpg files:

public void decryptFile(InputStream in, InputStream secKeyIn, InputStream pubKeyIn, char[] pass) throws IOException, PGPException, InvalidCipherTextException {
    Security.addProvider(new BouncyCastleProvider());

    PGPPublicKey pubKey = readPublicKeyFromCol(pubKeyIn);

    PGPSecretKey secKey = readSecretKeyFromCol(secKeyIn, pubKey.getKeyID());

    in = PGPUtil.getDecoderStream(in);

    JcaPGPObjectFactory pgpFact;


    PGPObjectFactory pgpF = new PGPObjectFactory(in, new BcKeyFingerprintCalculator());

    Object o = pgpF.nextObject();
    PGPEncryptedDataList encList;

    if (o instanceof PGPEncryptedDataList) {

        encList = (PGPEncryptedDataList) o;

    } else {

        encList = (PGPEncryptedDataList) pgpF.nextObject();

    }

    Iterator<PGPPublicKeyEncryptedData> itt = encList.getEncryptedDataObjects();
    PGPPrivateKey sKey = null;
    PGPPublicKeyEncryptedData encP = null;
    while (sKey == null && itt.hasNext()) {
        encP = itt.next();
        secKey = readSecretKeyFromCol(new FileInputStream("PrivateKey.asc"), encP.getKeyID());
        sKey = secKey.extractPrivateKey(new BcPBESecretKeyDecryptorBuilder(new BcPGPDigestCalculatorProvider()).build(pass));
    }
    if (sKey == null) {
        throw new IllegalArgumentException("Secret key for message not found.");
    }

    InputStream clear = encP.getDataStream(new BcPublicKeyDataDecryptorFactory(sKey));

    pgpFact = new JcaPGPObjectFactory(clear);

    PGPCompressedData c1 = (PGPCompressedData) pgpFact.nextObject();

    pgpFact = new JcaPGPObjectFactory(c1.getDataStream());

    PGPLiteralData ld = (PGPLiteralData) pgpFact.nextObject();
    ByteArrayOutputStream bOut = new ByteArrayOutputStream();

    InputStream inLd = ld.getDataStream();

    int ch;
    while ((ch = inLd.read()) >= 0) {
        bOut.write(ch);
    }

    //System.out.println(bOut.toString());

    bOut.writeTo(new FileOutputStream(ld.getFileName()));
    //return bOut;

}

public static void encryptFile(OutputStream out, String fileName, PGPPublicKey encKey) throws IOException, NoSuchProviderException, PGPException {
    Security.addProvider(new BouncyCastleProvider());

    ByteArrayOutputStream bOut = new ByteArrayOutputStream();

    PGPCompressedDataGenerator comData = new PGPCompressedDataGenerator(PGPCompressedData.ZIP);

    PGPUtil.writeFileToLiteralData(comData.open(bOut), PGPLiteralData.BINARY, new File(fileName));

    comData.close();

    PGPEncryptedDataGenerator cPk = new PGPEncryptedDataGenerator(new BcPGPDataEncryptorBuilder(SymmetricKeyAlgorithmTags.TRIPLE_DES).setSecureRandom(new SecureRandom()));

    cPk.addMethod(new BcPublicKeyKeyEncryptionMethodGenerator(encKey));

    byte[] bytes = bOut.toByteArray();

    OutputStream cOut = cPk.open(out, bytes.length);

    cOut.write(bytes);

    cOut.close();

    out.close();
}

Now here is how to invoke/run the above:

try {
         decryptFile(new FileInputStream("encryptedFile.gpg"), new FileInputStream("PrivateKey.asc"), new FileInputStream("PublicKey.asc"), "yourKeyPassword".toCharArray());

        PGPPublicKey pubKey = readPublicKeyFromCol(new FileInputStream("PublicKey.asc"));

        encryptFile(new FileOutputStream("encryptedFileOutput.gpg"), "fileToEncrypt.txt", pubKey);




    } catch (PGPException e) {
        fail("exception: " + e.getMessage(), e.getUnderlyingException());
    }

Thank you and +1 for taking the time to add insight to this old question. — CodeMed, Oct 19 '15 at 16:57

score 2 · Answer 2 · answered Feb 11 '15 at 23:55

2

It just means that content has been signed and then encrypted, the routine provided does not know how to deal with it, but at least tells you that. PGP protocol presents as a series of packets some of which can be wrapped in other ones (for example compressed data can also wrap signed data or simply literal data, these can be used to generate encrypted data as well, actual content always appears in literal data).

If you look at the verifyFile method in the SignedFileProcessor in the Bouncy Castle OpenPGP examples package you will see how to handle the signature data and get to the literal data containing the actual content.

I would also recommend looking at RFC 4880 so you have some idea of how the protocol works. The protocol is very loose and both GPG, BC, and the variety of products out there reflect this - that said the looseness does mean that if you try and cut and paste your way to a solution you'll end up with a disaster. It's not complicated, but understanding is required here as well.

answered Feb 11 '15 at 23:55

David Hook

531
3
3

Thank you. I very much want to understand. But I need more specifics. Can you please show code to fix this? I am good at decomposing working code samples. This material is complicated. Your paragraphs seem esoteric without working code to illustrate what you mean. – CodeMed Feb 11 '15 at 23:59
I added the code for `SignedFileProcessor.verifyFile()` to my edit at the end of my OP. – CodeMed Feb 12 '15 at 00:43
The issue here is with understanding the PGP specification (argh!). I have now sent a message that is encoded but not signed and used a (slightly modified) version of your code to decode it. However, when the message is signed, the PGP object stream that you want to get at (with the LiteralData objects) is effectively 'nested' within the CompressedData object. We need to be totally clear what you want to do - have you already got the attachment part of your email out of the raw email, or do you want to fully decode the email exactly as it arrives? – J Richard Snape Feb 12 '15 at 10:38
Basically - I think you really want to save off the encrypted attachment and then decode it as a file. This suits the model of your `decryptFile()` code better and I *think* might suit your purposes better. I think this question is getting rather long for SO. Maybe we should continue it in chat? – J Richard Snape Feb 12 '15 at 10:41

Getting BouncyCastle to decrypt a GPG-encrypted message

2 Answers2

Linked