Decode a string in Java

Question

How do I properly decode the following string in Java

http%3A//www.google.ru/search%3Fhl%3Dru%26q%3Dla+mer+powder%26btnG%3D%u0420%A0%u0421%u045F%u0420%A0%u0421%u2022%u0420%A0%u0421%u2018%u0420%u040E%u0420%u0453%u0420%A0%u0421%u201D+%u0420%A0%u0420%u2020+Google%26lr%3D%26rlz%3D1I7SKPT_ru

When I use URLDecoder.decode() I the following error

java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: "u0"

Thanks, Dave

@Johan if it's part of a larger URL (like `http://foo.com/?url=`) it could be but otherwise, agreed — Daniel DiPaolo, Mar 23 '11 at 16:35
@Johan, why not? @Daniel, exactly my thoughts: http://www.google.com/search?q=http%3A//www.google.ru/search%3Fhl%3Dru%26q%3Dla+mer+powder%26btnG%3D%u0420%A0%u0421%u045F%u0420%A0%u0421%u2022%u0420%A0%u0421%u2018%u0420%u040E%u0420%u0453%u0420%A0%u0421%u201D+%u0420%A0%u0420%u2020+Google%26lr%3D%26rlz%3D1I7SKPT_ru — OscarRyz, Mar 23 '11 at 16:35
To add to what @Johan said, it should start like "http://www.google.ru/search?hl=ru&q=la+mer+powder" The colon and question mark in this pattern are part of the URI specification and cannot be encoded. The encoding is used to escape a colon or question mark (or other characters) within a name or value which is part of the URL. — chiccodoro, Mar 23 '11 at 16:35
Although, of course this whole string presented by Dave could be a value within a single argument in a different URL... — chiccodoro, Mar 23 '11 at 16:36

score 2 · Answer 1 · answered Mar 23 '11 at 16:36

According to Wikipedia, "there exist a non-standard encoding for Unicode characters: %uxxxx, where xxxx is a Unicode value". Continuing: "This behavior is not specified by any RFC and has been rejected by the W3C".

Your URL contains such tokens, and the Java URLDecoder implementation doesn't support those.

score 2 · Answer 2 · answered Mar 23 '11 at 16:39

%uXXXX encoding is non-standard, and was actually rejected by W3C, so it's natural, that URLDecoder does not understand it.

You can make small function, which will fix it by replacing each occurrence of %uXXYY with %XX%YY in your encoded string. Then you can procede and decode the fixed string normally.

score 1 · Answer 3 · edited May 23 '17 at 12:05

After having had a good look at the solution presented by @ariy I created a Java based solution that is also resilient against encoded characters that have been chopped into two parts (i.e. half of the encoded character is missing). This happens in my usecase where I need to decode long urls that are sometimes chopped at 2000 chars length. See What is the maximum length of a URL in different browsers?

public class Utils {

    private static Pattern validStandard      = Pattern.compile("%([0-9A-Fa-f]{2})");
    private static Pattern choppedStandard    = Pattern.compile("%[0-9A-Fa-f]{0,1}$");
    private static Pattern validNonStandard   = Pattern.compile("%u([0-9A-Fa-f][0-9A-Fa-f])([0-9A-Fa-f][0-9A-Fa-f])");
    private static Pattern choppedNonStandard = Pattern.compile("%u[0-9A-Fa-f]{0,3}$");

    public static String resilientUrlDecode(String input) {
        String cookedInput = input;

        if (cookedInput.indexOf('%') > -1) {
            // Transform all existing UTF-8 standard into UTF-16 standard.
            cookedInput = validStandard.matcher(cookedInput).replaceAll("%00%$1");

            // Discard chopped encoded char at the end of the line (there is no way to know what it was)
            cookedInput = choppedStandard.matcher(cookedInput).replaceAll("");

            // Handle non standard (rejected by W3C) encoding that is used anyway by some
            // See: https://stackoverflow.com/a/5408655/114196
            if (cookedInput.contains("%u")) {
                // Transform all existing non standard into UTF-16 standard.
                cookedInput = validNonStandard.matcher(cookedInput).replaceAll("%$1%$2");

                // Discard chopped encoded char at the end of the line
                cookedInput = choppedNonStandard.matcher(cookedInput).replaceAll("");
            }
        }

        try {
            return URLDecoder.decode(cookedInput,"UTF-16");
        } catch (UnsupportedEncodingException e) {
            // Will never happen because the encoding is hardcoded
            return null;
        }
    }
}

ariy · Answer 4 · 2011-07-25T09:32:03.657

we started with Vartec's solution but found out additional issues. This solution works for UTF-16, but it can be changed to return UTF-8. The replace all is left for clarity reasons and you can read more at http://www.cogniteam.com/wiki/index.php?title=DecodeEncodeJavaScript

static public String unescape(String escaped) throws UnsupportedEncodingException
{
    // This code is needed so that the UTF-16 won't be malformed
    String str = escaped.replaceAll("%0", "%u000");
    str = str.replaceAll("%1", "%u001");
    str = str.replaceAll("%2", "%u002");
    str = str.replaceAll("%3", "%u003");
    str = str.replaceAll("%4", "%u004");
    str = str.replaceAll("%5", "%u005");
    str = str.replaceAll("%6", "%u006");
    str = str.replaceAll("%7", "%u007");
    str = str.replaceAll("%8", "%u008");
    str = str.replaceAll("%9", "%u009");
    str = str.replaceAll("%A", "%u00A");
    str = str.replaceAll("%B", "%u00B");
    str = str.replaceAll("%C", "%u00C");
    str = str.replaceAll("%D", "%u00D");
    str = str.replaceAll("%E", "%u00E");
    str = str.replaceAll("%F", "%u00F");

    // Here we split the 4 byte to 2 byte, so that decode won't fail
    String [] arr = str.split("%u");
    Vector<String> vec = new Vector<String>();
    if(!arr[0].isEmpty())
    {
        vec.add(arr[0]);
    }
    for (int i = 1 ; i < arr.length  ; i++) {
        if(!arr[i].isEmpty())
        {
            vec.add("%"+arr[i].substring(0, 2));
            vec.add("%"+arr[i].substring(2));
        }
    }
    str = "";
    for (String string : vec) {
        str += string;
    }
    // Here we return the decoded string
    return URLDecoder.decode(str,"UTF-16");
}

Decode a string in Java

4 Answers4