What's the proper way to escape URL variables with Spring's RestTemplate when calling a Spring RestController?

Question

When calling RestTemplate.exchange to do a get request, such as:

String foo = "fo+o";
String bar = "ba r";
restTemplate.exchange("http://example.com/?foo={foo}&bar={bar}", HttpMethod.GET, null, foo, bar)

what's the proper to have the URL variables correctly escaped for the get request?

Specifically, how do I get pluses (+) correctly escaped because Spring is interpreting as spaces, so, I need to encode them.

I tried using UriComponentsBuilder like this:

String foo = "fo+o";
String bar = "ba r";
UriComponentsBuilder ucb = UriComponentsBuilder.fromUriString("http://example.com/?foo={foo}&bar={bar}");
System.out.println(ucb.build().expand(foo, bar).toUri());
System.out.println(ucb.build().expand(foo, bar).toString());
System.out.println(ucb.build().expand(foo, bar).toUriString());
System.out.println(ucb.build().expand(foo, bar).encode().toUri());
System.out.println(ucb.build().expand(foo, bar).encode().toString());
System.out.println(ucb.build().expand(foo, bar).encode().toUriString());
System.out.println(ucb.buildAndExpand(foo, bar).toUri());
System.out.println(ucb.buildAndExpand(foo, bar).toString());
System.out.println(ucb.buildAndExpand(foo, bar).toUriString());
System.out.println(ucb.buildAndExpand(foo, bar).encode().toUri());
System.out.println(ucb.buildAndExpand(foo, bar).encode().toString());
System.out.println(ucb.buildAndExpand(foo, bar).encode().toUriString());

and that printed:

http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba r
http://example.com/?foo=fo+o&bar=ba r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba r
http://example.com/?foo=fo+o&bar=ba r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r
http://example.com/?foo=fo+o&bar=ba%20r

The space is correctly escaped in some instances, but the plus is never escaped.

I also tried UriTemplate like this:

String foo = "fo+o";
String bar = "ba r";
UriTemplate uriTemplate = new UriTemplate("http://example.com/?foo={foo}&bar={bar}");
Map<String, String> vars = new HashMap<>();
vars.put("foo", foo);
vars.put("bar", bar);
URI uri = uriTemplate.expand(vars);
System.out.println(uri);

with the exact same result:

http://example.com/?foo=fo+o&bar=ba%20r

Use UriTemplate and call expand on it. It will return a string with the url correctly escaped. — Shay Elkayam, May 20 '18 at 08:08

score 19 · Accepted Answer · answered May 22 '18 at 15:14

Apparently, the correct way of doing this is by defining a factory and changing the encoding mode:

String foo = "fo+o";
String bar = "ba r";
DefaultUriBuilderFactory factory = new DefaultUriBuilderFactory();
factory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.VALUES_ONLY);
URI uri = factory.uriString("http://example.com/?foo={foo}&bar={bar}").build(foo, bar);
System.out.println(uri);

That prints out:

http://example.com/?foo=fo%2Bo&bar=ba%20r

This is documented here: https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#web-uri-encoding

I think this solution can be applied for [this question](https://stackoverflow.com/questions/48906034). — Steffen Harbich, May 29 '18 at 06:41

score 11 · Answer 2 · edited Oct 07 '21 at 07:59

I think your problem here is that RFC 3986, on which UriComponents and by extension UriTemplate are based, does not mandate the escaping of + in a query string.

The spec's view on this is simply:

sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
                  / "*" / "+" / "," / ";" / "="

pchar       = unreserved / pct-encoded / sub-delims / ":" / "@"

query       = *( pchar / "/" / "?" )

URI         = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

If your web framework (Spring MVC, for example!) is interpreting + as a space, then that is its decision and not required under the URI spec.

With reference to the above, you will also see that !$'()*+,; are not escaped by UriTemplate. = and & are escaped because Spring has taken an "opinionated" view of what a query string looks like -- a sequence of key=value pairs.

Likewise, #[] and whitespace are escaped because they are illegal in a query string under the spec.

Granted, none of this is likely to be any consolation to you if you just quite reasonably want your query parameters escaped!

To actually encode the query params so your web framework can tolerate them, you could use something like org.springframework.web.util.UriUtils.encode(foo, charset).

The problem is that Spring is being inconsistent here. And if I pre-encode + as %2B by calling UriUtils.encode, then, the % gets further encoded by UriTemplate breaking the string. — Pablo Fernandez, May 22 '18 at 14:24
I don't think it's inconsistent. If you've already encoded your params, Spring can't know that. You can tell it you've already encoded them with e.g. `UriComponentsBuilder.fromUriString("http://example.com/?foo={foo}").queryParams(...).build(true).toUriString()` — ryanp, May 22 '18 at 14:47

score 7 · Answer 3 · answered May 22 '18 at 12:09

I'm starting to believe this is a bug and I reported here: https://jira.spring.io/browse/SPR-16860

Currently, my workaround is this:

String foo = "fo+o";
String bar = "ba r";
String uri = UriComponentsBuilder.
    fromUriString("http://example.com/?foo={foo}&bar={bar}").
    buildAndExpand(vars).toUriString();
uri = uri.replace("+", "%2B"); // This is the horrible hack.
try {
    return new URI(uriString);
} catch (URISyntaxException e) {
    throw new RuntimeException("UriComponentsBuilder generated an invalid URI.", e);
}

which is a horrible hack that might fail in some situations.

See the answer I posted, this uses everything from URIBuilder and adds a workaround to encode and send the query components. — Tarun Lalwani, May 22 '18 at 13:36

score 6 · Answer 4 · answered May 22 '18 at 13:35

For this one I would still prefer the encoding to be resolved using a proper method instead of using a Hack like you did. I would just use something like below

String foo = "fo+o";
String bar = "ba r";
MyUriComponentsBuilder ucb = MyUriComponentsBuilder.fromUriString("http://example.com/?foo={foo}&bar={bar}");

UriComponents uriString = ucb.buildAndExpand(foo, bar);
// http://example.com/?foo=fo%252Bo&bar=ba+r
URI x = uriString.toUri();

// http://example.com/?foo=fo%2Bo&bar=ba+r
String y = uriString.toUriString();

// http://example.com/?foo=fo%2Bo&bar=ba+r
String z = uriString.toString();

And of course the class is like below

class MyUriComponentsBuilder extends UriComponentsBuilder {
    protected UriComponentsBuilder originalBuilder; 

    public MyUriComponentsBuilder(UriComponentsBuilder builder) {
        // TODO Auto-generated constructor stub
        originalBuilder = builder;
    }


    public static MyUriComponentsBuilder fromUriString(String uri) {
        return new MyUriComponentsBuilder(UriComponentsBuilder.fromUriString(uri));
    }


    @Override
    public UriComponents buildAndExpand(Object... values) {
        // TODO Auto-generated method stub
        for (int i = 0; i< values.length; i ++) {
            try {
                values[i] = URLEncoder.encode((String) values[i], StandardCharsets.UTF_8.toString());
            } catch (UnsupportedEncodingException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
        return originalBuilder.buildAndExpand(values);
    }

}

Still not a cleanest possible way but better then doing a hardcoded replace approach

Prakash Ayappan · Answer 5 · 2018-05-22T16:37:27.853

2

You could use UriComponentsBuilder in Spring (org.springframework.web.util.UriComponentsBuilder)

String url = UriComponentsBuilder
    .fromUriString("http://example.com/")
    .queryParam("foo", "fo+o")
    .queryParam("bar", "ba r")
    .build().toUriString();

restTemplate.exchange(url , HttpMethod.GET, httpEntity);

edited May 22 '18 at 16:37

answered May 20 '18 at 09:09

Prakash Ayappan

455
2
8

Do you mean `foo` instead of `"{foo}"`? – Pablo Fernandez May 21 '18 at 10:17
You can just use the variables `foo` and `bar` like I did in my question. Another difference is that I'm using a template in my question. Not sure if this is relevant, but using a template with `UriCompontentsBuilder`, it seems not to escape the `+`. – Pablo Fernandez May 22 '18 at 08:37
See updated answer. UriComponentsBuilder will appropriately escape – Prakash Ayappan May 22 '18 at 16:39
Does not work for me. If I output with `System.out.println(url);`, I get `http://example.com/?foo=fo+o&bar=ba r` – Leo Mekenkamp Sep 15 '22 at 11:53
Adding `.encode()` after (or before) `.build()` will give me `http://example.com/?foo=fo+o&bar=ba%20r` – Leo Mekenkamp Sep 15 '22 at 11:59

What's the proper way to escape URL variables with Spring's RestTemplate when calling a Spring RestController?

5 Answers5

Linked