1

I am trying to convert many addresses to latitudes/longitudes using the httr package so I downloaded the map data from datasciencetoolkit.org and now am submitting a rather long POST/GET request to a local server at "http://localhost:8080/street2coordinates"

Here is the relevant code:

library(httr)
data2 <- paste0("[",paste(paste0("\"",data$address,"\""),collapse=","),"]")
url  <- "http://localhost:8080/street2coordinates"
response <- POST(url,body=data2)
json     <- fromJSON(content(response,type="text"))

For example, data2 has about 100,000 addresses in it. Is there a way to monitor how many requests have been sent as the request is being processed so that I can have some idea of when it will complete?

Thanks!

garson
  • 1,505
  • 3
  • 22
  • 56
  • you are doing a `POST` that contains all the addresses in it, so there aren't multiple submissions. One way would be to look at the logs on the DSTK side. You'll see each one there as they process. – hrbrmstr Jun 27 '15 at 14:53
  • http://stackoverflow.com/questions/22887833/r-how-to-geocode-a-simple-address-using-data-science-toolbox – user227710 Jun 27 '15 at 15:11
  • 1
    @user227710 the OP knows how to use the DSTK. They just want to be able to get a status of what's going on. – hrbrmstr Jun 27 '15 at 15:15
  • Hi @hrbrmstr can you tell me how I can see the logs locally? Thanks! – garson Jun 28 '15 at 00:37

1 Answers1

1

I assume you're using the httr package (if you are, please add this to your question). In any case, you can put up a progress bar for both up- and downloads by supplying progress() as an argument to GET and POST like this:

data2 <- toJSON(data$address)
url  <- "http://localhost:8080/street2coordinates"
response <- POST(url, config = progress(), body = data2)
json     <- fromJSON(content(response, type = "text"))

If ynou want to do something different to the standard progress bar you could look at the code for progress and see that it creates a set of configuration parameters like this (noting type can be "up", "down" or the default c("up", "down"):

config(noprogress = FALSE, progressfunction = progress_bar(type))

You could in turn look at httr:::progress_bar's code to look at how it works and write your own if you so desired.

Nick Kennedy
  • 12,510
  • 2
  • 30
  • 52
  • Thanks for responding. I tried progress() per your suggestion but it seems to only show 100% when the progress is completed and doesn't show progress interim, which is really what I'm looking for. Thank you for the suggestion though - let me know if you have other ideas. Thanks again! – garson Jun 27 '15 at 23:23
  • @garson it's been fixed in the latest GitHub release as per https://github.com/hadley/httr/pull/252 – Nick Kennedy Jun 28 '15 at 16:56