3

This works as expected - it returns one record:

https://terraref.ncsa.illinois.edu/bety/api/v1/search.json?sitename=~Season%204&limit=1

this does not

when I use the base url terraref.org rather than terraref.ncsa.illinois.edu:

https://terraref.org/bety/api/v1/search.json?sitename=~Season%204&limit=1

because in this second case the % is replaced by %25 and the query term is then not found.

Questions

  • How can I fix it?
  • Why has this appeared in the last few months and

Update

you can see question history for the red herring related to the R packages I was using thanks to the comments, I've narrowed down the issue, I think

I've included tags ruby-on-rails and nginx because these are the tools that the api and server are using.

David LeBauer
  • 31,011
  • 31
  • 115
  • 189
  • urlencoding maybe? – cory Feb 11 '20 at 00:01
  • @cory not sure if this is what you mean, but `URLencode('Foo%20Bar')` returns `Foo%20Bar` – David LeBauer Feb 11 '20 at 00:06
  • Verify that it is not a server issue and/or misconfigured server redirect as it was in this case https://stackoverflow.com/a/59779232/2864740 - in that case the issue was reproducible without r/httr. – user2864740 Feb 11 '20 at 00:10
  • It seems like that is what is happening... twice. https://www.w3schools.com/tags/ref_urlencode.ASP – cory Feb 11 '20 at 00:11
  • 3
    %20 is code for space, %25 is code for % – cory Feb 11 '20 at 00:11
  • 1
    @user2864740 I have done; removing 25 from the string makes it work as expected – David LeBauer Feb 11 '20 at 00:11
  • 1
    So, if instead of passing `"Test 1"`, you pass `URLencode("Test 1")`... does it work? – cory Feb 11 '20 at 00:21
  • @cory no it doesn’t :-( – David LeBauer Feb 11 '20 at 00:27
  • cant reproduce the error in your MRE on R 3.6.2. Is it `utils:::modifyList` you are using? because i cant see that the function calls any other function than itself. My best guess is also the url encoding as mentioned in the first two comments: Check `URLencode("Foo%20Bar", reserved = TRUE, repeated = TRUE)` and `URLencode("Foo%20Bar", reserved = TRUE, repeated = FALSE)`. So one hypothesis would be that the `repeated` parameter was changed in a function call or at least passed through over `...`,...but i cant see where your MRE code comes to an encoding function,... – Tonio Liebrand Mar 05 '20 at 09:33
  • related: https://stackoverflow.com/questions/59779173/httr-replaces-with-25-in-url-sometimes?rq=1. But as you seem to use https, probably not the issue for you. But maybe of interest for future googlers who arrive here =). – Tlatwork Mar 05 '20 at 14:40
  • *the function modifyList* IS NOT *replacing %20 with %2520* on my system (R 3.6.3). – s_baldur Mar 05 '20 at 16:21
  • @sindri_baldur I can confirm that the error in my MRE is not working on R 3.6.1 as well. I'll update the question – David LeBauer Mar 06 '20 at 21:27
  • As @cory mentioned % = %25, have you tried escaping the %? so passing foo\%20bar instead of foo%20bar – Beavatron Prime Mar 10 '20 at 12:52

2 Answers2

4

Somehow, you are encoding the value for sitename 2 times. In this example the decoded value is Season 4 when you encode first time Season%204and doing it the second time %20Season%25204.

Investigate the code to verify whether value for sitename already encoded and you doing it again.

luizinhoab
  • 435
  • 3
  • 8
4

Check Chrome DevTools to debug the problem.

enter image description here

Your call is being redirected with the following query parameters.

sitename: ~Season 4
limit: 1

If you choose the encoded view, you'll see the culprit.

enter image description here

So, as one might expect, it's just as simple as double url encoding.

If you need to use terraref.org as a base url, replace space with a plus.

https://terraref.org/bety/api/v1/search.json?sitename=~Season+4&limit=1
Yevgen
  • 1,576
  • 1
  • 15
  • 17
  • 1
    @David LeBauer If you are interested in more justification about space/plus solution, check out [this](https://stackoverflow.com/a/29948396/3262990) amazing answer. – Yevgen Mar 11 '20 at 16:17