Weird String format when decrypting

Question

I am trying to program a little challenge response app. My idea is to enter a text, encrypt it, show it as a QR code so the other side can scan it, decrypt it and show me the original text. But the final text is in a strange format and i can't figure out whats wrong. Here is what I am doing right now:

The 'challenge' string is the text from a TextView

byte[] secret = encrypt(publicKey, challenge.getBytes(StandardCharsets.UTF_8));
challengeenc = bytes2String(secret);

public byte[] encrypt(PublicKey key, byte[] plaintext) throws NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException
{
    Cipher enc = Cipher.getInstance("RSA");
    enc.init(Cipher.ENCRYPT_MODE, key);
    return enc.doFinal(plaintext);
}

private static String bytes2String(byte[] bytes)
{
    StringBuilder string = new StringBuilder();
    for (byte b : bytes) {
        String hexString = Integer.toHexString(0x00FF & b);
        string.append(hexString.length() == 1 ? "0" + hexString : hexString);
    }
    return string.toString();
}

Then the QR code part comes to transform and read it, but this also has to work since the challengeenc is equal to the string I read with the QR code scanner (challenge)

byte[] loesung = decrypt(privateKey, challenge.getBytes());
loesungstr = bytes2String(loesung);

Now the loesungstr should be equal to the string I started with but right now I entered "aaaaaaaa" and the loesungstr is: "1a3e3aa2e6a804fe6035c3f2879219ff971f119dee7e513f9311a27c3e7a7d58a27f826c45c5ace3699bc11ce7176a1ee3c9f4c1190402af5a79bbd8944750c85e73e4cda459ff904156186e1c010ea861a8f0be78594c00e22d049213b381de8afd877877ae9cf59169c77f088fe552a0e552260ed68f599858eaf1585916128778758db5c6d8efe32844d208932289f86202c10485533cdd89f1bd08d6e86f81c201d7d707ba435a5472a36cd7dc7584156000004b4a7a8fa320d3f3a919f9f901d86b965ba09a452372fd35cf84fc3a1841c18196c735c76b2ce75bf8bd03a671d42f6c0bac39eabf56ea09b37d00176f1b45a3917f81226ef2f0a312e515"

I tried a lot of stuff and converted this string from hex to string, from hex to ASCII, ... but I am not able to figure out where it went wrong..

Does somebody have an idea what this string might be?

PS: Since I can sign a text and verify the signature I know the key pair I created is correct

Just as an assumption, shouldn't you encrypt with PRIVATE key and decrypt with PUBLIC key (I see reverse in your code)? — lospejos, Dec 04 '16 at 11:24
@lospejos if I encrypt with the PRIVATE key, everybody will be able to decrypt it using the PUBLIC key.. so why would I then encrypt it.. (signing is encryption using the private key, so everybody can verify the signature with the public key) — daniel_mdf, Dec 04 '16 at 11:30
@zaph because the idea behind this challenge-response is to prove that the public key belongs to the person by decrypting with the private key. — daniel_mdf, Dec 04 '16 at 11:46
You are encrypting the plaintext and then hex-encoding, and then decrypting the hex-encoding and then hex-encoding that. It does not make sense. You shoul hex-*de*-code the hex-encoded cipher text, and then decrypt it to recover the original plaintext. — user207421, Dec 04 '16 at 11:58
@zaph The trouble with that is the key distribution problem, which doesn't exist in PKI as the public key is, err, public. Could you possibly stick to the point? — user207421, Dec 04 '16 at 11:59
@zaph He is using public and private keys, which don't have a key distribution problem either. There is no evidence here that he isn't using PKI on top of that. The problem here will not be solved by switching to symmetric encryption. That will only introduce another problem. You are off topic. — user207421, Dec 04 '16 at 12:04
Please show us the complete code, especially the `decrypt` method. — Roland Illig, Dec 04 '16 at 12:48
@daniel_mdf Try it without the hex-encoding, just to verify whether the problem is there or in the crypto code. — user207421, Dec 04 '16 at 21:40

score 2 · Answer 1 · answered Dec 04 '16 at 12:54

Your code should follow this basic pattern, which I just tested successfully.

In particular, make sure that every variable has a good name. Maybe this is where your confusion comes from.

import javax.crypto.Cipher;
import java.nio.charset.StandardCharsets;
import java.security.GeneralSecurityException;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.PrivateKey;
import java.security.PublicKey;

public class Rsa {

    static KeyPair generateKeyPair() throws GeneralSecurityException {
        KeyPairGenerator gen = KeyPairGenerator.getInstance("RSA");
        gen.initialize(4096);
        return gen.generateKeyPair();
    }

    static byte[] encrypt(byte[] plaintext, PublicKey key) throws GeneralSecurityException {
        Cipher cipher = Cipher.getInstance("RSA");
        cipher.init(Cipher.ENCRYPT_MODE, key);
        return cipher.doFinal(plaintext);
    }

    static byte[] decrypt(byte[] ciphertext, PrivateKey key) throws GeneralSecurityException {
        Cipher cipher = Cipher.getInstance("RSA");
        cipher.init(Cipher.DECRYPT_MODE, key);
        return cipher.doFinal(ciphertext);
    }

    public static void main(String[] args) throws GeneralSecurityException {
        KeyPair keys = generateKeyPair();
        byte[] plaintext = "hello, world".getBytes(StandardCharsets.UTF_8);
        byte[] ciphertext = encrypt(plaintext, keys.getPublic());
        byte[] plainAgain = decrypt(ciphertext, keys.getPrivate());
        System.out.println(new String(plainAgain, StandardCharsets.UTF_8));
    }
}

score 0 · Answer 2 · edited May 23 '17 at 12:24

0

you should have another method that decode the encoded string, i.e hexStringToByteArray() before decrypting it:

this method is from this answer:

public static byte[] hexStringToByteArray(String s) {
    int len = s.length();
    byte[] data = new byte[len / 2];
    for (int i = 0; i < len; i += 2) {
        data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
                             + Character.digit(s.charAt(i+1), 16));
    }
    return data;
}

and you use it when decrypting as below:

byte[] loesung = decrypt(privateKey, hexStringToByteArray(challenge));

to get the original plain text:

String plaintext = new String(loesung);

edited May 23 '17 at 12:24

Community

1
1

answered Dec 04 '16 at 12:36

Yazan

6,074
1
19
33

yeah, in the meantime I already implemented the exact method but it still doesn't work. For now the I do this: byte[] secret2 = encrypt(publicKey, challenge.getBytes(StandardCharsets.UTF_8)); challengeenc = bytes2String(secret2); and afterwards: byte[] loesung = decrypt(privateKey, hexStringToByteArray(challenge)); String loesungstr = new String(loesung); – daniel_mdf Dec 04 '16 at 13:04
@daniel_mdf thats weird. unless the 2 methods is not working on the same approach, thus the decode process does not match the original bytes created by `encrypt()`. you can avoid all this by using `Base64` using `encode()` and `decode()` this way you will neutralize the bytes-string issue – Yazan Dec 04 '16 at 13:09

Weird String format when decrypting

2 Answers2