URL encoding using the new Spring UriComponentsBuilder

Question

I'm attempting to use spring's UriComponentsBuilder to generate some urls for oauth interaction. The query parameters include such entities as callback urls and parameter values with spaces in them.

Attempting to use UriComponentBuilder (because UriUtils is now deprecated)

UriComponentsBuilder urlBuilder = UriComponentsBuilder.fromHttpUrl(oauthURL);
urlBuilder.queryParam("client_id", clientId);
urlBuilder.queryParam("redirect_uri", redirectURI);
urlBuilder.queryParam("scope", "test1 test2");

String url = urlBuilder.build(false).encode().toUriString();

Unfortunately, while the space in the scope parameter is successfully replaced with '+', the redirect_uri parameter is not at all url encoded.

E.g,

redirect_uri=https://oauth2-login-demo.appspot.com/code

should have ended up

redirect_uri=https%3A%2F%2Foauth2-login-demo.appspot.com%2Fcode

but was untouched. Diving into the code, specifically org.springframework.web.util.HierarchicalUriComponents.Type.QUERY_PARAM.isAllowed(c) :

if ('=' == c || '+' == c || '&' == c) {
  return false;
}
else {
  return isPchar(c) || '/' == c || '?' == c;
}

clearly allows ':' and '/' characters, which by gum, it shouldn't. It must be doing some other type of encoding, though for the life of me, I can't imagine what. Am I barking up the wrong tree(s) here?

Thanks

score 32 · Accepted Answer · edited Nov 18 '22 at 19:03

32

UriComponentsBuilder is encoding your URI in accordance with RFC 3986, with section 3.4 about the 'query' component of a URI being of particular note.

Within the 'query' component, the characters / and : are permitted, and do not need escaping.

To take the / character for example: the 'query' component (which is clearly delimited by unescaped ? and (optionally) # characters), is not hierarchical and the / character has no special meaning. So it doesn't need encoding.

edited Nov 18 '22 at 19:03

Ondra Žižka

43,948
41
217
277

answered Jan 30 '14 at 15:36

simonh

529
4
4

8

This is not correct, because `&` and other characters with meaning are also not escaped. UriComponentsBuilder is not url-encoding the query parameter. – Adam Millerchip Nov 07 '17 at 04:30
@Adam Millerchip, I don't understand - surely & _is_ getting escaped. The OP pasted the code snippet snippet showing that & isn't allowed, nor is = or +. All of these would get escaped. – simonh Jan 09 '18 at 19:56
2

You would think so, but it doesn't. Try it and see. – Adam Millerchip Jan 10 '18 at 00:46
1

I tried it did indeed escape & and =. Not sure why + wasn't escaped, given the code snippet in the OP. I suspect my original reasoning holds: the + symbol doesn't mean anything within the query fragment. UriComponentsBuilder urlBuilder = UriComponentsBuilder.fromHttpUrl("http://example.org"); urlBuilder.queryParam("scope", "test1&test2=test3+test4"); String url = urlBuilder.build(false).encode().toUriString(); System.out.println(url); Result: `http://example.org?scope=test1%26test2%3Dtest3+test4` – simonh Jan 11 '18 at 10:35
(sorry, poor S.O. formatting in my above comment - the 'example.org' actually has an http:// prefix but it got stripped out after posting. – simonh Jan 11 '18 at 10:42
It does escape all four of [& + = space] with this code: ``` UriComponentsBuilder builder = UriComponentsBuilder .fromUri(uri) .queryParam("content-type", "mimeType&q +=w?/") .queryParam("name", "name"); URI uri2 = builder.buildAndExpand(new HashMap<>()).encode().toUri(); ``` – Velizar Hristov Jul 10 '18 at 09:49
Either way, you should be calling encode() before build() in most cases, as the javadoc clearly states. – rougou Dec 05 '18 at 09:21
I said to call encode() before build() as `toUriString()` does for you but found out the hard way this doesn't work if your query string contains curly braces, which are treated as special placeholders for variables. For example, `param1={{{{` causes an exception, and `param1={{{}` just doesn't get encoded. Strangely, `toUriString()` used to do build() then encode() but they changed it and didn't even update the Javadoc for that method. I doubt they have tested it, either. – rougou Dec 28 '18 at 02:17
8

After I reported the toUriString() behavior, a spring developer promptly fixed it to use build().encode().toUriString() (the original behavior) if no variables are present. Now query parameters like "{}" will correctly be encoded. https://jira.spring.io/browse/SPR-17630 – rougou Jan 10 '19 at 04:14
I pass a String `foo+bar` to the `UriComponentsBuilder#queryParam()`, and query it using `RestTemplate`. On the other side, I have a Spring endpoint with `@RequestParam("someName") someName: String`, and yet, that gets `foo bar`. I consider that broken. – Ondra Žižka Nov 18 '22 at 19:10

score 27 · Answer 2 · answered Jan 21 '14 at 13:13

27

from what I understand, UriComponentsBuilder doesn't encode the query parameters automatically, just the original HttpUrl it's instantiated with. In other words, you still have to explicitly encode:

String redirectURI= "https://oauth2-login-demo.appspot.com/code";
urlBuilder.queryParam("redirect_uri", URLEncoder.encode(redirectURI,"UTF-8" ));

answered Jan 21 '14 at 13:13

Black

5,023
6
63
92

1

Well... the 'encode' method states: Encodes all URI components using their specific encoding rules, and returns the result as a new {@code UriComponents} instance. This seems to imply it does URL encoding. It seems to leave it to 'Type' (in this case Type.QUERY_PARAM) to decide which characters to encode. So it will encode some characters.. but not some very important ones. What does the encode method do if not encode query params for URL encoding? – ticktock Jan 28 '14 at 00:29
2

it encodes the *URL* you pass to it, but not each query param – Black Jan 28 '14 at 01:08
21

Well.. that's not awfully useful. It's odd because it's a URL builder and you add query params, THEN build and THEN encode. I would have assumed it was building me a safe URL. What's the point of adding query params if they don't get encoded with the URL? – ticktock Jan 30 '14 at 19:03
2

I agree, and would have assumed as you did. Perhaps the designer has some rationale for not implicitly encoding params, but also maybe not – Black Jan 31 '14 at 01:09
5

UriComponentsBuilder contains an encode() method, which I think will encode both the url and the query params. – ashario Mar 21 '17 at 00:27
1

From the documentation of urlbuilder.encode() "Request to have the URI template pre-encoded at build time, and URI variables encoded separately when expanded". So using .encode() seems to be a nice finishing touch. – XDS Feb 17 '20 at 07:22
I gave it up to use UriComponentsBuilder::encode as well (':' for e.g. are not encoded, SPACE becomes %20, ..). I now encode queryParams explicitely as described above and use: builder.build(true).toUri() to provide the complete correctly encoded URL – phirzel Oct 03 '20 at 09:15

score 4 · Answer 3 · answered Apr 09 '18 at 07:43

4

Try to scan the UriComponentsBuilder doc, there is method named build(boolean encoded)

Sample code 1:

UriComponents uriComponents = UriComponentsBuilder.fromPath("/path1/path2").build(true);

Here is my sample code 2:

UriComponents uriComponents = UriComponentsBuilder.newInstance()
            .scheme("https")
            .host("my.host")
            .path("/path1/path2").query(parameters).build(true);

URI uri= uriComponents.toUri();

ResponseEntity<MyEntityResponse> responseEntity = restTemplate.exchange(uri,
            HttpMethod.GET, entity, typeRef);

answered Apr 09 '18 at 07:43

Andoy Abarquez

1,119
4
17
30

2

`.build(true)` alone wont escape the `.queryParam`, it just says that all your params are already escaped. Feels useles since it doesn't do any escaping... probably work in progress from Spring side – jediz Jun 02 '18 at 08:53
3

@jediz It is useful if you have a url that is already encoded and don't want it to be verified and encoded again. Maybe they should rename it from `encoded` to `alreadyEncoded` – rougou Dec 28 '18 at 02:25

score 0 · Answer 4 · answered Jan 31 '22 at 14:03

I tried all the solutions above until I got it working.

In my example, I was trying to encode the ZonedDateTime format 2022-01-21T10:17:10.228+06:00. The plus sign was a problem.

What solved my issue was encoding the value manually + using URI instead of the string value (both were very important).

Before:

restTemplate.exchange(
  UriComponentsBuilder
    .queryParam("fromDateTime", "2022-01-21T10:17:10.228+06:00")
    .build()
    .toUriString(),
  HttpMethod.GET,
  null,
  new ParameterizedTypeReference<List<MyDto>>() {}
);

After:

restTemplate.exchange(
  UriComponentsBuilder
    .queryParam("fromDateTime", URLEncoder.encode("2022-01-21T10:17:10.228+06:00", StandardCharsets.UTF_8))
    .build(true)
    .toUri(),
  HttpMethod.GET,
  null,
  new ParameterizedTypeReference<List<MyDto>>() {}
);

score 0 · Answer 5 · answered Jun 18 '23 at 05:56

I put encode() before build() to make query parameter encoding work for me. These tests compare with vs without the encode() call.

import static org.assertj.core.api.Assertions.assertThat;

import org.junit.jupiter.api.Test;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.util.UriComponentsBuilder;

public class EncodeQueryParametersTest {
    @Test
    public void withoutEncode() {
        final MultiValueMap<String, String> queryParameters = new LinkedMultiValueMap<>();
        queryParameters.add("fullname", "First Last");
        assertThat(UriComponentsBuilder.newInstance().queryParams(queryParameters).build().getQuery()).isEqualTo("fullname=First Last");
    }

    @Test
    public void withEncode() {
        final MultiValueMap<String, String> queryParameters = new LinkedMultiValueMap<>();
        queryParameters.add("fullname", "First Last");
        assertThat(UriComponentsBuilder.newInstance().queryParams(queryParameters).encode().build().getQuery()).isEqualTo("fullname=First%20Last");
    }
}

URL encoding using the new Spring UriComponentsBuilder

5 Answers5

Linked

Related