How to correctly encode and decode a string in Base64?

Question

I want to encode a string in Base64 for later decoding it. I encode it doing this:

public static String encryptString(String string) {     
    byte[] bytesEncoded = Base64.getEncoder().encode(string.getBytes());
    return (new String(bytesEncoded));
}

Then, the encoded string is stored on disk using UTF-8. After restarting the application, the encoded string is readed from disk and I'm trying to decode the string using this:

public static String decryptString(String string) {
    byte[] valueDecoded = Base64.getDecoder().decode(string);
    return (new String(valueDecoded));
}

Something is wrong because it is giving me this exception:

java.lang.IllegalArgumentException: Illegal base64 character d
at java.base/java.util.Base64$Decoder.decode0(Base64.java:743)
at java.base/java.util.Base64$Decoder.decode(Base64.java:535)
at java.base/java.util.Base64$Decoder.decode(Base64.java:558)

This is a TRACE step by step

1º i encode this: {"configuration":{"shop":{"name":"","addressLine1":"","addressLine2":"","postalCode":"","city":"","country":"","phoneNumber":""}},"jointBets":[],"groups":[{"name":"Test","members":[]}]}

into this: eyJjb25maWd1cmF0aW9uIjp7InNob3AiOnsibmFtZSI6IiIsImFkZHJlc3NMaW5lMSI6IiIsImFkZHJlc3NMaW5lMiI6IiIsInBvc3RhbENvZGUiOiIiLCJjaXR5IjoiIiwiY291bnRyeSI6IiIsInBob25lTnVtYmVyIjoiIn19LCJqb2ludEJldHMiOltdLCJncm91cHMiOlt7Im5hbWUiOiJUZXN0IiwibWVtYmVycyI6W119XX0=

2º i store it on disk in utf8

3º i retreive it from disk and it's this string:

eyJjb25maWd1cmF0aW9uIjp7InNob3AiOnsibmFtZSI6IiIsImFkZHJlc3NMaW5lMSI6IiIsImFkZHJlc3NMaW5lMiI6IiIsInBvc3RhbENvZGUiOiIiLCJjaXR5IjoiIiwiY291bnRyeSI6IiIsInBob25lTnVtYmVyIjoiIn19LCJqb2ludEJldHMiOltdLCJncm91cHMiOlt7Im5hbWUiOiJUZXN0IiwibWVtYmVycyI6W119XX0=

4º i decode it and get the exception.

why don't you `encodeToString`? but tried your code (jshell) worked fine — user85421, Mar 08 '19 at 18:32
I assume `.decode(string)` should be `.decode(string.getBytes())` as well. — Ervin Szilagyi, Mar 08 '19 at 18:36
maybe you could post the data that is generating that error (I wonder how d is an invalid Base64 character???) ((changing default charset between runs?) — user85421, Mar 08 '19 at 18:39
`getBytes()` could be a problem because it uses your system's local default character set. Which could be different on different systems. — markspace, Mar 08 '19 at 18:48
or `new String(byte[])` that uses the system default charset while `decode` uses ISO8859-1 (despite this shouldn't have any influence on Base64 characters, or? UTF16LE/BE) — user85421, Mar 08 '19 at 18:55
just tested with `new String(bytesEncoded, UTF_16)` and got similar exception when decoding.... don't happen if using `encodeToString` (uses ISO8859-1) — user85421, Mar 08 '19 at 19:10
btw i'm storing the encoded string into disk using utf-8 before decoding it, later i'm reading it from disk. Maybe that's the problem? how to solve it? — NullPointerException, Mar 08 '19 at 19:19
check the strings (bytes) being written to and read from disk... > *"maybe you could post the data that is generating that error"* — user85421, Mar 08 '19 at 19:21
Possible duplicate of [Base64 Encoding in Java](https://stackoverflow.com/questions/13109588/base64-encoding-in-java) — Mikhail Kholodkov, Mar 08 '19 at 19:25
@MikhailKholodkov no, delete your possible duplicat erequests and read the coments — NullPointerException, Mar 08 '19 at 19:26
@CarlosHeuberger i added the trace on the questions with the data — NullPointerException, Mar 08 '19 at 19:32
I checked the source of decode, `d` is the hexadecimal value (`Integer.toString(char, 16)`) of the offending character, in this case probably a carriage return which is outside the base64 alphabet and shouldn't be there - try the MIME encoders/decoders of Base64 (or avoid the carriage return/linefeeds) — user85421, Mar 08 '19 at 19:33
can't understand how to do that.. please can you post a sample? — NullPointerException, Mar 08 '19 at 19:34
`Base64.getMimeEncoder()` `...getMimeDecoder()` (or/and change how writing/reading from disk) — user85421, Mar 08 '19 at 19:36
omg it's working with that. What's the explanation about this? can you writte an answer with your solution and explaining why that solve the issue? i will accept it — NullPointerException, Mar 08 '19 at 19:45
instead of `Base64.getEncoder()` you write `Base64.getMimeEncoder()` - these ignore invalid characters: *"**MIME** ... All line separators or other characters not found in the base64 alphabet table are ignored in decoding operation"* — user85421, Mar 08 '19 at 19:46
@CarlosHeuberger ignored? then they will be lost? in that case this is useless :S — NullPointerException, Mar 08 '19 at 19:48
the ignored characters are NOT part of the encoded data - they are just *noise* - if this is a problem you should NOT write/read the extra carriage return (maybe you are using `println`??? but, after 8 years of SO you should know about [mcve] ) — user85421, Mar 08 '19 at 19:49
what do you mean? For example, i want to add this: "España" which contains a character that isn't in the base64 alphabet. What will happen with that character? will it be lost when encoded ? so after decoded should it be "Espaa" ? — NullPointerException, Mar 08 '19 at 19:50
you should first encode it, to obtain base64 (characters), that is "RXNwYfFh" for above string - somehow I have the impression that you do not know what Base64 is for — user85421, Mar 08 '19 at 19:52
I can't understand you. Please, can you explain it with my sample? i want to add this: "España" which contains a character that isn't in the base64 alphabet. What will happen with that character? will it be lost when encoded ? so after decoded should it be "Espaa" ? — NullPointerException, Mar 08 '19 at 19:53
"España" sure is not Base64, it must be encoded first - what **you** are doing with first part of **your** program - would result in "RXNwYfFh" - but if you now (for example, I must assume since we dont know it) use `println` it will have an additional linefeed added to the end (the `ln` part of the method) which is not allowed in plain base64 — user85421, Mar 08 '19 at 19:55
if then use the MIME decoder it will ignore the extra linefeed and still return "España" if feed with "RXNwYfFh\n" - have you tried it to see if it is returning "Espaa"??? — user85421, Mar 08 '19 at 19:58
then, can the base64 decoded text be different from the original non encoded text in any situation? — NullPointerException, Mar 08 '19 at 20:01
WHY are you using Base64? it will not be different as long as it not changed between encoding and decoding — user85421, Mar 08 '19 at 20:02
@CarlosHeuberger because i need to store the string in a non human readable condition, so, encrypted or encoded — NullPointerException, Mar 08 '19 at 20:09
then Base64 is not the indicated one...non-readable maybe, encrypt sure not. almost all *nix system have a command to encode/decode it It is just a method to encode **binary** data using ASCII character that can safely be transferred between system (e.g. in older days by E-Mail or forums). — user85421, Mar 08 '19 at 20:12
@CarlosHeuberger this is a java application used in Windows OS, so i think can't have that easy function — NullPointerException, Mar 08 '19 at 20:14
not to hard in windows (powershell has it, if I am not wrong, I use it a lot in git-bash on windows) e.g. https://www.igorkromin.net/index.php/2017/04/26/base64-encode-or-decode-on-the-command-line-without-installing-extra-tools-on-linux-windows-or-macos/ or online https://www.base64encode.org/ — user85421, Mar 08 '19 at 20:15
I'm considering using google Tink library, but it seems that is too much complex for just encrypt and decrypt a string — NullPointerException, Mar 08 '19 at 20:17
@CarlosHeuberger please whould you post an answer with your solution of getMime... ? I'll mark as accepted — NullPointerException, Mar 11 '19 at 12:33

score 3 · Answer 1 · edited Jun 19 '19 at 10:05

The old Base64 utility add linebreaks every 76 characters in Java8. The result looks like that:

/9j/4AAQSkZJRgABAgAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0a
HBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIy
MjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCABkAGQDASIA
AhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQA
AAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3
ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWm
...

It seems that this behaviour changed with some version. At least with Java11 the decoder is not accepting line-breaks anymore. To avoid the problem you could change you method

public static String decryptString(String string) {
    byte[] valueDecoded = Base64.getDecoder().decode(string.replace("\n","").replace("\r","");
    return new String(valueDecoded);
}

I used `base64 -w 0` and despite having no linebreaks I still got that error. Only changing Base64.getDecoder() to Base64.getMimeDecoder() helped, I don't really know why. — user158037, Aug 16 '21 at 12:49

Mikhail Kholodkov · Answer 2 · 2019-03-08T19:57:01.703

Then, the encoded string is stored on disk using UTF-8. After restarting the application, the encoded string is readed from disk and I'm trying to decode the string using this:

This seems to be a point of failure. Most likely your problem is OS/JDK dependent Apparently the following code seems to work well for me (Win 7, latest JDK 1.8):

public static void main(String[] args) throws IOException {
    String source = "{\"configuration\":{\"shop\":{\"name\":\"España\",\"addressLine1\":\"\",\"addressLine2\":\"\"," +
                "\"postalCode\":\"\",\"city\":\"\",\"country\":\"\",\"phoneNumber\":\"\"}},\"jointBets\":[]," +
                "\"groups\":[{\"name\":\"Test\",\"members\":[]}]}";

    // Encode string
    String encoded = encryptString(source);

    System.out.println("Base64 encoded: " + encoded);

    // Temp Dir
    String tempDir = System.getProperty("java.io.tmpdir");

    // Write to File
    try (BufferedWriter writer = new BufferedWriter(new FileWriter(tempDir + "data.txt"))) {
        writer.write(encoded);
    }

    // Read from File
    Path path = Paths.get(tempDir + "data.txt");

    Stream<String> lines = Files.lines(path);
    String dataFromFile = lines.collect(Collectors.joining("\n"));
    lines.close();

    // Compare content
    assert encoded.equals(dataFromFile);

    // Decode string
    String decoded = decryptString(dataFromFile);
    System.out.println("Base64 decoded: " + decoded);
}

public static String encryptString(String string) {
    byte[] bytesEncoded = Base64.getEncoder().encode(string.getBytes(StandardCharsets.UTF_8));
    return new String(bytesEncoded);
}

public static String decryptString(String string) {
    byte[] valueDecoded = Base64.getDecoder().decode(string);
    return new String(valueDecoded);
}

Base64 encoded: eyJjb25maWd1cmF0aW9uIjp7InNob3AiOnsibmFtZSI6IkVzcGHDsWEiLCJhZGRyZXNzTGluZTEiOiIiLCJhZGRyZXNzTGluZTIiOiIiLCJwb3N0YWxDb2RlIjoiIiwiY2l0eSI6IiIsImNvdW50cnkiOiIiLCJwaG9uZU51bWJlciI6IiJ9fSwiam9pbnRCZXRzIjpbXSwiZ3JvdXBzIjpbeyJuYW1lIjoiVGVzdCIsIm1lbWJlcnMiOltdfV19

Base64 decoded: {"configuration":{"shop":{"name":"España","addressLine1":"","addressLine2":"","postalCode":"","city":"","country":"","phoneNumber":""}},"jointBets":[],"groups":[{"name":"Test","members":[]}]}

his problem is reading a carriage return (code `d`) probably at the end of the text, that is not allowed in plain base64 — user85421, Mar 08 '19 at 19:48

score 0 · Answer 3 · answered Mar 08 '19 at 19:00

My guess is that you are not specifying a charset. Try running the below maybe with and without the charset specified for the String constructor to verify.

   @Test
    public void base64Test() throws Exception{
        String string = "ABCDF";

        byte[] bytesEncoded = Base64.getEncoder().encode(string.getBytes());
        String encodedStr = (new String(bytesEncoded,Charset.forName("ISO-8859-1")));
        System.out.println(encodedStr);

        byte[] valueDecoded = Base64.getDecoder().decode(encodedStr);
        String decodedStr = (new String(valueDecoded,Charset.forName("ISO-8859-1")));
        System.out.println(decodedStr);
    }

first part is exactly what `encodeToString` does (as commented/suggested/asked above) — user85421, Mar 08 '19 at 19:07

How to correctly encode and decode a string in Base64?

3 Answers3