httr replaces "%" with "%25" in URL sometimes

Question

When using httr::GET, in certain queries it replaces % with safe representation %25, but in other queries it doesn't. I cannot find any rule that would make this happen.

I'm using httr 1.4.1

Sample query where % is replaced (notice the error code and that URL entered is not the same as in response object returned):

> httr::GET("jira.spring.io/rest/api/latest/search?jql=project=Spring%20Framework&startAt=0")
Response [https://jira.spring.io/rest/api/latest/search?jql=project=Spring%2520Framework&startAt=0]
  Date: 2020-01-16 22:57
  Status: 400
  Content-Type: application/json;charset=UTF-8
  Size: 196 B

Query where it is not replaced (no error, URL in response same as entered):

> httr::GET("issues.jenkins-ci.org/rest/api/latest/search?jql=project='WEBSITE'%20OR%20project='Infrastructure'&startAt=0")
Response [https://issues.jenkins-ci.org/rest/api/latest/search?jql=project='WEBSITE'%20OR%20project='Infrastructure'&startAt=0]
  Date: 2020-01-16 23:02
  Status: 200
  Content-Type: application/json;charset=UTF-8
  Size: 430 kB

What is going on? Is it a bug in httr? Or should I change some parameters in GET() call?

I may have run into a issue but came up with a different underlying cause - the `modifyList` function replaces `%` with `%25` https://stackoverflow.com/q/60160208/199217 — David LeBauer, Feb 11 '20 at 17:47

user2864740 · Accepted Answer · 2020-01-16T23:23:10.067

4

tldr; use HTTPS requests with jira.spring.io to avoid a broken protocol upgrade.

It's not an R/HTTR issue. It's the website. Compare the results of HTTP ("failing with mystery %25") and HTTPS ("succeeding"):

http://jira.spring.io/rest/api/latest/search?jql=project=Spring%20Framework&startAt=0

{"errorMessages":["Error in the JQL Query: The character '%' is a reserved JQL character. You must enclose it in a string or use the escape '\u0025' instead. (line 1, character 15)"],"errors":{}}
https://jira.spring.io/rest/api/latest/search?jql=project=Spring%20Framework&startAt=0

{"errorMessages":["Error in the JQL Query: Expecting either 'OR' or 'AND' but got 'Framework'. (line 1, character 16)"],"errors":{}}

There appears to be a 'malfunction' in the HTTP -> HTTPS redirect protocol upgrade, which has this response header:

Status Code: 301 Moved Permanently
Location: https://jira.spring.io/rest/api/latest/search?jql=project=Spring%252520Framework&startAt=0
                                                                          ^^^^^

Thus a solution is to use the HTTPS endpoint and avoid the strange target Location..

edited Jan 16 '20 at 23:23

answered Jan 16 '20 at 23:18

user2864740

60,010
15
145
220

I don't understand a lot about https but would this be consistent with the same or similar problem I found here: https://stackoverflow.com/q/60160208/199217? – David LeBauer Feb 11 '20 at 17:48
The issue above also affects “normal” browsers, let it follow the 302. This case above is *explicitly* related to a redirect.. although it does seems similar. :| I’ll see how Postman works with the same 302 later .. if/when I remember. – user2864740 Feb 11 '20 at 18:51
@DavidLeBauer In the case above, it is indeed an error in the jira.spring.io site; either that or Chrome, IE, and Postman are all "wrong" :-) I wonder if r/httr might have had support (perhaps even accidentally) to *"auto fixup"* such URLs that has since been removed? Also, consider that this question *only* applies for an *invalid 30x response* redirect. The original query sent is correct. – user2864740 Feb 11 '20 at 23:13

httr replaces "%" with "%25" in URL sometimes

1 Answers1

Linked