Encoding and decoding byte[] with ZXing

Question

I am developping an Android app, and I need to encode and decode a bytes array in a QRCode generated with the ZXing app. My problem is that my message decoded does not exactly match the generated byte array. I tried to create a QRCode based on a byte array containing incrementing indexes, i.e.

input = [0, 1, 2, ..., 124, 125, 126, 127, -128, -127,... -3, -2, -1, 0, 1, 2, ...]

And after encoding the message in the QRCode and decoding it on the responder side, I obtain the following byte array output:

output = [0, 1, 2, ..., 124, 125, 126, 127, 63, 63,... 63, 63, 63, 0, 1, 2, ...]

All the "negative" byte values are turned to ASCII char 63: '?' question mark characters. I assume that something is going wrong with the encoding charset, but since I am using ISO-8859-1 which everyone claims to be the solution of such kind of issue (other topic treating the same kind of issue or here), I don't see where is my mistake, or if I am skipping a step during the instanciation of the encoding or the decoding. Here is the code that I execute to encode a given byte array:

String text = "";
byte[] res = new byte[272];
for (int i = 0; i < res.length; i++) {
    res[i] = (byte) (i%256);
}
try {
    text = new String(res, "ISO8859_1");
} catch (UnsupportedEncodingException e) {
    // TODO
}
Intent intent = new Intent(Intents.Encode.ACTION);
Intent.addFlags(Intent.FLAG_ACTIVITY_CLEAR_WHEN_TASK_RESET);
intent.putExtra(Intents.Encode.TYPE, Contents.Type.TEXT);
intent.putExtra(Intents.Encode.FORMAT, "ISO8859_1");
intent.putExtra(Intents.Encode.DATA, text);
intent.putExtra(Intents.Encode.FORMAT, BarcodeFormat.QR_CODE.toString());

boolean useVCard = intent.getBooleanExtra(USE_VCARD_KEY, false);
QRCodeEncoder qrCodeEncoder = new QRCodeEncoder(activity, intent, dimension, useVCard);
Bitmap bitmap = qrCodeEncoder.encodeAsBitmap();

And to decode a QRCode, I send the following Intent

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.qrcodeDecoding);

    Intent intent = new Intent(Intents.Scan.ACTION);
    intent.putExtra(Intents.Scan.MODE, Intents.Scan.QR_CODE_MODE);
    startActivityForResult(intent, 0);
}

And wait for Result:

@Override
protected void onActivityResult(int request, int result, Intent data)
{
    if(request == 0)
    {
        //action
        if(result == RESULT_OK)
        {
            String res = data.getStringExtra(Intents.Scan.RESULT);
            byte[] dat = null;

            try{
                    dat = res.getBytes("ISO8859_1");
            } catch(UnsopportedEncodingException e) {
                    //TODO
            }
        }
        else if(result == RESULT_CANCELED)
        {
            //TODO
        }
    }

}

Could you please tell me where are my mistakes, or where should I look at?

Thank you a lot,

Franck

Answer 1

In one of my apps I needed to encode and decode a bytes array in a QRCode generated with the ZXing app. As the byte array contained compressed text data I wanted to avoid base64 encoding. It is possible to do this but as I have so far not seen a complete set of code snippets I will post them here.

Encoding:

public void showQRCode(Activity activity, byte[] data){
  Intent intent = new Intent("com.google.zxing.client.android.ENCODE");
  intent.putExtra("ENCODE_TYPE", "TEXT_TYPE");
  intent.putExtra("ENCODE_SHOW_CONTENTS", false);
  intent.putExtra("ENCODE_DATA", new String(data, "ISO-8859-1"));
  activity.startActivity(intent);
}

Start scanning:

public static void startQRCodeScan(Activity activity){
  Intent intent = new Intent(com.google.zxing.client.android.SCAN);
  intent.putExtra("SCAN_MODE", "QR_CODE_MODE");
  intent.putExtra("CHARACTER_SET", "ISO-8859-1");
  activity.startActivityForResult(intent, 0);
}

Scan result handler:

public void onActivityResult(int requestCode, int resultCode, Intent intent) {
  byte[] result = intent.getStringExtra("SCAN_RESULT").getBytes("ISO-8859-1");
  ...
}

I think not setting CHARACTER_SET to ISO-8859-1 in the intent data for starting the scan is the point that made the code of the original question fail. It took me quite some time to dig this out as I have not seen this clearly posted anywhere and Latin 1 encoding is the standard encoding for QR code in Xzing. Especially tricky is the fact that the Xzing online decoder http://zxing.org/w/decode.jspx does not set CHARACTER_SET as well so that the generated QR code looks faulty when decoded on this site.

Answer 2

You are making the mistake of thinking that you can turn arbitrary binary data into a valid string without using some kind of armouring. It doesn't work. Binary -> text -> binary is lossy using any of the standard character sets / encoding. (Hint: using UTF-8 won't work either.)

You should use something like base64 encoding or hexadecimal encoding to ensure that the binary data doesn't get mangled.

Answer 3

Conceptually, QR codes encode text, not bytes. Inside of course they translate input to a series of bytes, though that's opaque to the caller. You are right that, as it happens, choosing the right encoding would let you sneak the bytes through, and ISO-8859-1 is the right choice here. It does work, actually.

ASCII is not possible since it does not define chars for >= 128, and UTF-8 is definitely not going to work

The issue here is probably your code. I am not sure what you're attempting here... it looks like you're setting up to send an Intent somewhere (to Barcode Scanner?) but then you don't, you're just making an Intent and sending it to some code you copied from the project? I imagine something's gone wrong with how you are setting extras to the Intent.

This should be much simpler if you're doing it within your app. Just reuse QRCodeEncoder.encodeAsBitmap() directly and delete the rest of this.