Avoid Double Encoding of URL query param with Spring's RestTemplate

Question

I'm trying to use Spring's RestTemplate::getForObject to make a request for a URL which has a URL query param.

I've tried:

Using a string
Creating a URI with URI::new
Creating a URI with URI::create
Using UriComponentsBuilder to build the URI

No matter which of these I use, encoding the url query param with URLEncoder::encode gets double encoded and using this encoding leaves the url query param unencoded.

How can I send this request without double encoding the URL? Here's the method:

try {
    UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(detectUrl)
            .queryParam("url", URLEncoder.encode(url, "UTF-8"))
            .queryParam("api_key", "KEY")
            .queryParam("api_secret", "SECRET");
    URI uri = builder.build().toUri();
    JSONObject jsonObject = restTemplate.getForObject(uri, JSONObject.class);
    return jsonObject.getJSONArray("face").length() > 0;
} catch (JSONException | UnsupportedEncodingException e) {
    e.printStackTrace();
}

Here's an example:

Without URLEncoder:

http://www.example.com/query?url=http://query.param/example&api_key=KEY&api_secret=SECRET

With URLEncoder:

http://www.example.com/query?url=http%253A%252F%252Fquery.param%252Fexample&api_key=KEY&api_secret=SECRET

':' should be encoded as %3A and '/' should be encoded as %2F. This does happen - but then the % is encoded as %25.

Please provide an example URI and how it gets double encoded. — Sotirios Delimanolis, Dec 16 '15 at 20:26
@SotiriosDelimanolis I've added an example of URI double encoding. — cscan, Dec 16 '15 at 20:32
I don't believe the URL specification requires `:` or `/` to be encoded. You should be able to use `http://www.example.com/query?url=http://query.param/example&api_key=KEY&api_secret=SECRET` directly. — Sotirios Delimanolis, Dec 16 '15 at 20:46

score 15 · Accepted Answer · edited Oct 07 '21 at 05:49

A UriComponentsBuilder is a builder for UriComponents which

Represents an immutable collection of URI components, mapping component type to String values.

The URI specification defines what characters are allowed in a URI. This answer summarizes the list to the characters

ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~:/?#[]@!$&'()*+,;=

By those rules, your URI

http://www.example.com/query?url=http://query.param/example&api_key=KEY&api_secret=SECRET

is perfectly valid and requires no additional encoding.

The method URLEncoder#encode(String, String)

Translates a string into application/x-www-form-urlencoded format using a specific encoding scheme.

That is not the same thing. That process is defined here, and URLEncoder (afaik) should be following it pretty closely.

In your original code, using URLEncoder#encode converted your input url into

http%3A%2F%2Fquery.param%2Fexample

The character % is invalid in a URI so must be encoded. That's what the UriComponents object constructed by your UriComponentsBuilder is doing.

This is unnecessary since your URI is completely valid to begin with. Get rid of the use of URLEncoder.

But if I have a parameter inside of the second url, then it will create problems right ? — omjego, Oct 22 '18 at 12:39

Avoid Double Encoding of URL query param with Spring's RestTemplate

1 Answers1

Linked