1

I've come across similar questions on Stack Overflow but nothing that quite does it for me. The closest example might be this one.

What I've been doing is sending API requests to the NASA POWER API. My requests look something like this:

"https://power.larc.nasa.gov/beta/api/temporal/hourly/point?start=20080101&end=20180101&latitude=-34.96260135&longitude=-54.9401847474085&community=re&parameters=ALLSKY_SFC_SW_DWN,T2M,WS2M&format=json&user=dataqueryabed&header=false&time-standard=lst"

I use GET to download the information. I get the 200 code, and the response object returned seems fine to my eyes. Now, when I download the data, I give it a longitude and latitude pair. I have generally been able to make this work, but sometimes it just doesn't.

I try to convert the JSON objects into dataframes like so:

#Step 1:   
content(some_request, as = "text")
#Step 2: 
some_response_unframed = fromJSON(rawToChar(response_given$content))
#Step 3: 
response_framed = enframe(unlist(some_response_unframed))

#What we want is "response framed"

The above seems to work sometimes, but not others, and I can't figure out why. For example, the coordinates latitude -34.9626, longitude=-54.9402, this works perfectly. However, if I try the request with coordinates longitude 39.288 and latitude -6.280, I am able to download everything properly and it works up until the "fromJSON" step:

tanzania_offshore_try_now = fromJSON(rawToChar(tanzania_offshore_data$content))
Error: lexical error: invalid char in json text.
                                       <!DOCTYPE HTML PUBLIC "-//IETF/
                     (right here) ------^

I can't figure out:

  1. Why it works for some pairs and not others
  2. How am I supposed to edit the JSON content as in the previous posted answer? I'm not reading from a file or anything at this stage, it's a "response" type object in R.

Thanks in advance!

EDIT Thanks to @MrFlick for pointing out the obvious,not sure how I missed it. So, it seeems that all I needed to do was re-download the data. I now have it all working. I'm puzzled why the requests seem to work one minute and not the other, my best guess is that their server doesn't want you to download too many requests that are too similar in too quick of a succession. Not sure I'm worried now that my scripts are working though. Thanks again @MrFlick.

Abed
  • 183
  • 7
  • 1
    If you try going to the URL for the one that doesn't work, which is presumably, https://power.larc.nasa.gov/beta/api/temporal/hourly/point?start=20080101&end=20180101&latitude=39.288&longitude=-6.280&community=re&parameters=ALLSKY_SFC_SW_DWN,T2M,WS2M&format=json&user=dataqueryabed&header=false&time-standard=lst you'll see that no data is returned. Just an error message. That error message isn't in a JSON format. You'll need to contact those that provide the API to find out why no data is being returned. – MrFlick Jul 26 '21 at 02:36
  • Hi, and thanks. Perhaps I should have been clearer, but: no, there is some data returned in some cases and I still get this message. I will update the question to show how. Thanks, though. – Abed Jul 26 '21 at 02:40
  • 2
    @MrFlick - the link you posted returns raw JSON data when I click on it so it seems it's a server-side issue, perhaps API is intermittently overloaded. – Ritchie Sacramento Jul 26 '21 at 02:40
  • @27ϕ9 Oh. Interesting. I got a 502 Proxy Error (The proxy server received an invalid response from an upstream server.) – MrFlick Jul 26 '21 at 02:42
  • Okay, on second thought you might be right. – Abed Jul 26 '21 at 02:50
  • 2
    I clicked a few more times and also saw the 502 Proxy Error. OP, you can build some error checking into your function and retry when it fails - use `jsonlite::validate()` to check what's been downloaded is valid JSON. – Ritchie Sacramento Jul 26 '21 at 02:54

1 Answers1

0

This question was answered with community support. I should have just been checking my response objects better.

Abed
  • 183
  • 7