1

I want to do almost exactly this question: Pulling data from an API response list

But the trick is, I'm going to have dozens of responses from this loop I asked about in another question.

I.e. I have this dataset:

df<-structure(list(PROTOCOL_ID = c(1, 22, 543, 421, 55, 6), PROTOCOL_NO = c("CTSU-E1234", 
"BRUOG-j-1234", "tp-P-bob61", "PFIZER-T", "Jimbo", 
"INCONGRUENCE"), LIBRARY = c("Non-Oncology", "Oncology", "Non-Oncology", 
"Oncology", "Oncology", "Non-Oncology")), row.names = c(NA, 6L), class = "data.frame")

And using this purrr code, I have been able to successfully GET and PUT dozens of responses at a time:

library(httr)

token<- "12345"
base <- "https://mywebsite.website.com"
endpoint <- "/website-api/rest/protocolManagementDetails/"

UpdateAccountNumbers <- function(protocol){
  
  call2 <- paste(base,endpoint, protocol, sep="") 
  
  call2 <- paste(base,endpoint, protocol, sep="")  
  
  httpResponse <- GET(call2, add_headers(authorization = token))
  results <- fromJSON(content(httpResponse, "text"))
  
  results$hospitalAccountNo <- results$internalAccountNo
  
  call2 <- paste(base,endpoint, protocol, sep="") 
  
  httpResponse <- PUT(
    call2, 
    add_headers(authorization = token), 
    body=results, encode = "json", 
    verbose()
  )
}

purrr::walk(df$PROTOCOL_ID, UpdateAccountNumbers)

It works perfectly as intended and loops through my dataframe, plugs the "protocol_id" variable into the loop, 'GET's the data from that Protocol, changes what I want to, and then 'PUT's the data back in. It does this perfectly. And honestly so far, every single attempt I've tried has worked great and I'll get several "204 No Content" responses which means it worked great.

enter image description here

But I'm sure that every once and a while it wont work perfectly for certain protocols and I might get a response like "400 Bad Request" or something. And I'd love to be able to keep track of that.

Ideally something like this:

enter image description here

Joe Crozier
  • 944
  • 8
  • 20
  • Joe, would using `tryCatch` to track different behavior (like 400 errors) do the job? https://stackoverflow.com/a/12195574/8400969 – Michael Roswell Aug 15 '22 at 13:55
  • @MichaelRoswell hmm, it definitely seems like it. Im having trouble imagining how to squish it into my code at the moment, but let me do some thinking/playing – Joe Crozier Aug 15 '22 at 13:59
  • 1
    Based on a quick glance, wrapping your `GET` and `PUT` lines with `tryCatch` and some conditional stuff to deal with different HTTP error messages might do it... – Michael Roswell Aug 15 '22 at 14:37
  • 1
    `status_code(response)` will get you the status – gaut Aug 15 '22 at 16:56
  • @gaut Thank you. I feel like with that suggestion I am getting very close. I found this https://rpubs.com/cliex159/867722 after some searching, but my brain is still just having a hard time figuring out how to put the status_code() into the function above. I understand that if I were to just do a GET and then do status_code(httpresponse) it would work, but being inside of the function and returning the codes into a dataframe is throwing me, any suggestions to point me in the right direction? – Joe Crozier Aug 15 '22 at 17:43
  • 1
    sure, will take a look – gaut Aug 15 '22 at 17:51

1 Answers1

1

We can use:

protocolid <- protocolnb <- library_names <- get_codes <- put_codes <- list()

UpdateAccountNumbers <- function(protocol){
  call2 <- paste(base,endpoint, protocol, sep="") 
  call2 <- paste(base,endpoint, protocol, sep="")  
  httpResponse_get <- GET(call2, add_headers(authorization = token))
  results <- fromJSON(content(httpResponse_get, "text"))
  
  results$hospitalAccountNo <- results$internalAccountNo
  
  call2 <- paste(base,endpoint, protocol, sep="") 
  
  httpResponse_put <- PUT(
    call2, 
    add_headers(authorization = token), 
    body=results, encode = "json", 
    verbose()
  )
  
  # save stats 
  protocolid <<- append(protocolid, protocol)
  protocolnb <<- append(protocolnb, df$PROTOCOL_NO[match(protocol, df$PROTOCOL_ID)])
  library_names <<- append(library_names, df$LIBRARY[match(protocol, df$PROTOCOL_ID)])
  get_codes <<- append(get_codes, status_code(httpResponse_get))
  put_codes <<- append(put_codes, status_code(httpResponse_put))
}
purrr::walk(df$PROTOCOL_ID, UpdateAccountNumbers)

allresults <- tibble('protocolid'=unlist(protocolid),'protocolnb'=unlist(protocolnb),'library_names'=unlist(library_names), 'get_codes'=unlist(get_codes), 'put_codes'=unlist(put_codes) )

Since the question is not reproducible, let me know if there are remaining errors.

gaut
  • 5,771
  • 1
  • 14
  • 45
  • The UpdateAccountNumbers function still works (and then purrr::walk of that, it works, clearly updates website) but when I run the final line of allresults<-tibble.... it says: "Error in unlist(httpResponse_get) : object 'httpResponse_get' not found" Sorry its not reproducible, I know how to create reprex's for simple data frames, not sure how I could create one for an API call – Joe Crozier Aug 15 '22 at 18:20
  • 1
    No worries, it's not always doable. the correct varnames are `get_codes` and `put_codes` - see edit – gaut Aug 15 '22 at 18:22
  • Thank you so much for all your help so far! Hopefully last thing: So everything 'works' in the sense there are no errors, but the allresults dataframe at the end is 0 observations of 0 variables. When I run the line "protocolid <- protocolnb <- libr..." it creates a bunch of lists of zero, and then i run the lines for the function and the purrr:walk and it seems to work. It updates the website and I can see it returning status of "204 no content" if I watch the console carefully. But then when I run the allresults... line it just turns into 0x0. – Joe Crozier Aug 15 '22 at 18:31
  • When I step through the function line by line, it seems to work (and save out to the lists). Like if I type protocolnb <- append(protocolnb, Oncology$PROTOCOL_NO[match(3, Oncology$PROTOCOL_ID)]) it'll work (3 in there since its one of the numbers that could passed to protocol), but for some reason wont work as part of the function.... – Joe Crozier Aug 15 '22 at 18:53
  • 1
    yea I kindof expected sth like this with `purrr::map`. You can either use a `for` loop rather than `map`, or you can use the global assignment `<<-` inside your function: `protocolid <<- append(...` – gaut Aug 15 '22 at 19:31
  • Global assignment worked great! – Joe Crozier Aug 15 '22 at 20:05