10

I have a strange issue with urlencoding a plus sign + as a query param for a request against an API. The API's documentation states:

The date has to be in the W3C format, e.g. '2016-10-24T13:33:23+02:00'.

So far so good, so I'm using this code (minimalized) to generate the url, using Spring's UriComponentBuilder:

DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ssX");
ZonedDateTime dateTime = ZonedDateTime.now().minusDays(1);
String formated = dateTime.format(formatter);

UriComponentsBuilder uriComponentsBuilder = UriComponentsBuilder.fromUriString(baseUrl);
uriComponentsBuilder.queryParam("update", formated);
uriComponentsBuilder.build();
String url = uriComponentsBuilder.toUriString();

The unencoded query would look like this:

https://example.com?update=2017-01-05T12:40:44+01

The encoded string results in:

https://example.com?update=2017-01-05T12:40:44%2B01

which is (IMHO) a correctly encoded query String. See the %2B replacing the + in +01 at the end of the query string.

Now, however, when I send the request against the API using the encoded url, I get an error saying the request could not be handled.

If however, I replace the %2B with a + before sending the request, it works:

url.replaceAll("%2B", "+");

From my understanding, the + sign is a replacement for a whitespace. So the url that the server really sees after decoding must be 

https://example.com?update=2017-01-05T12:40:44 01

  • Am I right with this assumption?

  • Is there anything I can do, other than contacting the API's owner to make it work using the correctly encoded query, other than strange non standard string replacements?

UPDATE:

According to the specification RFC 3986 (Section 3.4), the + sign in a query param doesn't need to be encoded.

3.4. Query

The query component contains non-hierarchical data that, along with data in the path component (Section 3.3), serves to identify a
resource within the scope of the URI's scheme and naming authority
(if any). The query component is indicated by the first question
mark ("?") character and terminated by a number sign ("#") character
or by the end of the URI.

Berners-Lee, et al. Standards Track [Page 23] RFC 3986 URI Generic Syntax
January 2005

  query       = *( pchar / "/" / "?" )

The characters slash ("/") and question mark ("?") may represent data within the query component. Beware that some older, erroneous implementations may not handle such data correctly when it is used as the base URI for relative references (Section 5.1), apparently
because they fail to distinguish query data from path data when
looking for hierarchical separators. However, as query components
are often used to carry identifying information in the form of
"key=value" pairs and one frequently used value is a reference to
another URI, it is sometimes better for usability to avoid percent-
encoding those characters.

According to this answer on stackoverflow, spring's UriComponentBuilder uses this specification, but appearently it doesn't really. So a new question would be, how to make UriComponentBuilder follow the specs?

Community
  • 1
  • 1
baao
  • 71,625
  • 17
  • 143
  • 203
  • Refer to the table provided by @Simon Tewsi http://stackoverflow.com/questions/575440/url-encoding-using-c-sharp – Praburaj Jan 06 '17 at 12:10
  • 2
    @Prabu, Your reference does not answer the question. OP wants to know why the service accepts only the unencoded plus sign, not how to encode or not, or what C# utilities to use. A quick check of encoding rules indicates that the circumstances under which reserved characters are or must be encoded has changed slightly, but I haven't seen anything to suggest that the plus sign, used as part of a formatted date string, shouldn't be encoded. – arcy Jan 06 '17 at 12:20
  • That's exactly what I think too @arcy – baao Jan 06 '17 at 12:21
  • @Michael, `UriComponentsBuilder.build()` has one version with an encoding flag and one version without. The one without, which you are using, does not say (in the version of javadoc I looked at) whether it defaults to encoded or not. You don't indicate whether you have seen the resulting strings or are just assuming they are the output. Is it possible it isn't encoding and you haven't realized it? – arcy Jan 06 '17 at 12:24
  • No, I logged them several times. @arcy I'm sure they are exactly as shown – baao Jan 06 '17 at 12:26
  • P.S. setting the encoding flag doesn't change it. As well as using normalize(), encode() doesn't change the behaviour. @arcy – baao Jan 06 '17 at 12:27
  • `urlBuilder.queryParam("redirect_uri", URLEncoder.encode(redirectURI,"UTF-8" ));` Try sepecifying the charset to UTF-8 – Praburaj Jan 06 '17 at 12:41
  • I commented lot before you made a edit to the post. You edited 9 mins ago and I posted my comment 10 mins ago. – Praburaj Jan 06 '17 at 12:51

4 Answers4

1

So it seems like spring's UriComponentBuilder encodes the whole url, setting the encoding flag to false in the build() method has no effect, because the toUriString() method allways encodes the url, as it calls encode() explicitly after build():

/**
 * Build a URI String. This is a shortcut method which combines calls
 * to {@link #build()}, then {@link UriComponents#encode()} and finally
 * {@link UriComponents#toUriString()}.
 * @since 4.1
 * @see UriComponents#toUriString()
 */
public String toUriString() {
    return build(false).encode().toUriString();
}

The solution for me (for now) is encoding what really needs to be encoded manually. Another solution could be (might require encoding too) getting the URI and work with that further on

String url = uriComponentsBuilder.build().toUri().toString(); // returns the unencoded url as a string
baao
  • 71,625
  • 17
  • 143
  • 203
  • This does not work when using `UriComponentsBuilder.fromUriString(baseUrl)` and this baseUrl already contains encoded query params. In that case `uriComponentsBuilder.build().toUri().toString();` does encode already encoded characters again. This is especially the case if you would use something like `ServletUriComponentsBuilder.fromCurrentRequest().build().toString()`. – Marcel Overdijk Apr 23 '18 at 07:08
  • Now it seems they have changed the behavior to call encode() first followed by build(). Which will cause `java.lang.IllegalArgumentException: Mismatched open and close braces` if you have a value like "{{{{" in one of your query strings. https://github.com/spring-projects/spring-framework/blob/master/spring-web/src/main/java/org/springframework/web/util/UriComponentsBuilder.java#L445 – rougou Dec 27 '18 at 10:09
1

you can use builder.build().toUriString()

This worked for me

Thanks

Sanjay
  • 89
  • 1
  • 13
0

Encoding 2017-01-05T12:40:44+01

gives you 2017-01-05T12%3A40%3A44%2B01

instead of 2017-01-05T12:40:44%2B01 as you suggested.

Maybe that's why the server is not being able to handle your request, it's a half encoded date.

cahen
  • 15,807
  • 13
  • 47
  • 78
0

In the org/springframework/web/util/HierarchicalUriComponents.java

QUERY_PARAM {
        @Override
        public boolean isAllowed(int c) {
            if ('=' == c || '+' == c || '&' == c) {
                return false;
            }
            else {
                return isPchar(c) || '/' == c || '?' == c;
            }
        }
    }

the character '+' is not allowed so it will be encoded

fan chao
  • 1