0

I am gathering Google Trends data using the R package gtrendsR. I am pulling data on searches for "politics" in every U.S. state between two dates. Here is the code.

politics1 <- gtrends(query = "politics", geo = c("AL", "AK", "AZ", "AR", "CA"), start_date = "2015-08-01", end_date = "2015-10-01")

The problem is that gtrendsR does not allow for more than 5 locations (states) in "geo" at once. But, of course, I need to do the above for all 50 states. How can I create loop (or something similar) so that I can easily run this code on all states, in segments of 5 at a time, and then combine all of these separate objects (politics1, politics2, politics3, etc.) into one dataframe? Any thoughts are appreciated, thanks!

hans91
  • 1
  • 2

1 Answers1

0

Haven't tested this as I don't have the authentication set up but this will loop through each state in the states vector and append the results to a dataframe called df which starts empty:

states <- c("AL", "AK", "AZ", "AR", "CA")

df <- data.frame()
for (i in 1:length(states)){
      try(rbind(df, gtrends(query = "politics", geo = states[i], start_date = "2015-08-01", end_date = "2015-10-01")), silent = TRUE)
    }

Could be smarter and do it 5 at a time but for this I doubt it matters. All you need to do is add the states to states.

Jacob
  • 3,437
  • 3
  • 18
  • 31
  • Thanks for this. This makes sense, but I did get the error message below. thoughts? Error: unexpected '}' in: " rbind(dataframe, gtrends(query = "politics", geo = states[i], start_date = "2015-08-01", end_date = "2015-10-01") }" – hans91 Dec 09 '16 at 19:00
  • sry, missing bracket. try now. – Jacob Dec 09 '16 at 19:01
  • Just noticed that and fixed. Now receiving this message: Error in rbind(dataframe, gtrends(query = "politics", geo = states[i], start_date = "2015-08-01", : cannot coerce type 'closure' to vector of type 'list' – hans91 Dec 09 '16 at 19:02
  • ah, issue with the initial dataframe. try again! – Jacob Dec 09 '16 at 19:03
  • got it. getting this now, but it seems like this might be a gtrends issue? Though not sure how. Error in rbind(deparse.level, ...) : invalid list argument: all variables should have the same length – hans91 Dec 09 '16 at 19:08
  • ok so we are naively assuming here that the results from the api will have uniform dimensions, the obvious problem is that perhaps one loop is returning nothing. Or it doesn't like getting 50 queries in rapid succession. Swap the middle line of loop for `print(gtrends(query = "politics", geo = states[i], start_date = "2015-08-01", end_date = "2015-10-01"))` and see what happens. – Jacob Dec 09 '16 at 19:12
  • Yep, it looks like it is a gtrends problem. I get the following error: "Error: Not enough search volume. Please change your search terms." It prints as far as it can go before a state returns this error. – hans91 Dec 09 '16 at 19:23
  • made an edit to answer that does a rudimentary attempt to catch the error. – Jacob Dec 09 '16 at 19:37
  • Try the print thing wrapped in the try() and see what happens. – Jacob Dec 09 '16 at 20:01
  • perhaps the answer is here: http://stackoverflow.com/questions/39568710/google-trends-in-r-error-after-few-tries. I might have hit a daily quota. – hans91 Dec 09 '16 at 20:17
  • Update: when I add Sys.sleep(10) to the loop with print(), everything works fine and the results for each state appear. I do not run into the "not enough search volume" as before. But when I add Sys.sleep(10) to the rbind() code, I get the same invalid list argument error. thoughts? – hans91 Dec 11 '16 at 21:24
  • I think I detected the issue with rbind(). GTrends returns multiple objects. The one that contains what I need, in my original code, would be politics1$trend. Should I edit your code so that I have rbind(politics1$trend, gtrends . . .)? – hans91 Dec 11 '16 at 21:57
  • if you can then feel free, otherwise put the working code in a comment and I'll do it. – Jacob Dec 12 '16 at 13:33
  • Still haven't been able to get it to work. When I run the below, and do not hit the quota, I still get a DF with zero rows/columns.politics1 <- data.frame() for (i in 1:length(states)){Sys.sleep(10); rbind(politics1, gtrends(query = "politics", geo = states[i], start_date = "2016-05-01", end_date = "2016-05-31")$trend) } – hans91 Dec 14 '16 at 05:56
  • what do you get from `gtrends(query = "politics", geo = states[1], start_date = "2016-05-01", end_date = "2016-05-31")$trend`? – Jacob Dec 14 '16 at 07:17
  • That works. This "print" code you provided also works after adding $trend to the end. – hans91 Dec 14 '16 at 19:39
  • The result for me is Description:df [0 × 0] – user3591356 Aug 25 '22 at 21:10