1

This is a bit of a meta-question, and I'll try to include all the relevant bits and bobs. Also, it may be a completely preposterous idea, so don't hate me if it's stupid :)

Anyhow. For the project I'm currently working on, I need to collecting heaps and heaps of data. Currently I'm collecting the data from twitter, later on I might be collecting data from other sources. I've set up R to run on a server and I've written a piece of code (a simple infinite repeat loop) that continuously collects data along some pre-defined parameters.

Since I started data collection 2 days ago the code hasn't failed. However, what if it does and I don't notice? I'm obviously not monitoring the data collection process constantly. Is there any way, maybe a package for R, or a creative piece of code, that could be built into the loop which would alert me to the fact that the code has failed, or perhaps just send me a daily status update?

I am aware that R is probably not the best way of getting these heaps of data from the web. As it happens, I'm a social scientist and trying to work my way through the whole computer science angle of using big data bit by bit. So, basically, I will try to properly learn Python in the future, as people keep telling me this is a more adequate tool for my endeavours. However, as it stands now, my code is working, and I'd like to make sure I don's miss out on an important development.

Thanks in advance for your help!

Gregor Thomas
  • 136,190
  • 20
  • 167
  • 294
nikUoM
  • 639
  • 1
  • 8
  • 18
  • are you using linux? if so you can use this `nohup echo "Rscript myscript.R" | at -m NOW` and the server will send you an email or text when you are done. check out this man page for more info http://linux.die.net/man/1/at – bjoseph Sep 28 '15 at 17:12
  • I've never done it with a text message, but I've combined `tryCatch` and `sendmailR` to send me an e-mail when an error occurs, and I've done it on Unix and Windows. I also know people who have written SAS code to send text messages in the event of a failure, though I don't recall the details. I think the first step in accomplishing this would be to decide how you want to send the message. Is there a web API you can use to do the trick? Do you want to send a text via an e-mail address? (all mobile carriers have e-mail equivalents for their phone numbers) – Benjamin Sep 28 '15 at 17:15
  • awesome stuff guys! thanks so much. I am using both Ubuntu, Mac OS X and Windows :D so all of it is applicable, thanks. Email is obviously great, the only advantage of having a text message would be that I could always get it, even when abroad. But, having said that, most of the time I'd have wifi anyway. – nikUoM Sep 28 '15 at 17:25
  • 1
    Here are some resources that might get you started. Some of them may not work now, but they could get the idea going for you. http://www.r-bloggers.com/sending-a-text-in-r/ ; http://stackoverflow.com/questions/6543070/send-a-text-message-from-r ; http://r.789695.n4.nabble.com/Send-SMS-out-of-R-td829926.html – Benjamin Sep 28 '15 at 17:32
  • In addition to Benjamin's links, this is perhaps an exact duplicate: [Using R to send tweets](http://stackoverflow.com/q/4252694/903061). Also see the [twitteR](https://cran.rstudio.com/web/packages/twitteR/) package. – Gregor Thomas Sep 28 '15 at 17:57
  • thanks again. this should definitely get me sorted. if you post it as answers ill give you some upvotes ;) – nikUoM Sep 28 '15 at 18:01

0 Answers0