How to identify url in tweet using R

Question

I want to identify those tweets containing URL in my twitter data set. For example, using the sign of "http://".

How to proceed it in R? for example the tweets texts are

  "@RainxDog @twitpic Please HELP #OccupyWallStreet and RT this video: http://t.co/vjwNR7TC"

  "@degamuna Please HELP #OccupyWallStreet and RT this video: http://t.co/vjwNR7TC"

score 3 · Accepted Answer · answered Jul 11 '12 at 15:06

3

You can use grep

if(length(grep("http://",data))>0){
 data[grep("http://",data)]
}

answered Jul 11 '12 at 15:06

shhhhimhuntingrabbits

7,397
2
23
23

score 3 · Answer 2 · edited May 23 '17 at 12:27

Your relatively simple question, hides something that is actually very tricky. In your two examples, the urls:

were of the form: http://t.cp/ - what about bit.ly links? What about https?
the urls appeared at the end of the tweet. What about urls in the middle or start of the tweet?

Construct a set of sample tweets and make sure that your regular expression works.

Basically, you need a regular expression. Stackoverflow questions to look at are:

These questions also contain links.

score 0 · Answer 3 · answered Jul 11 '12 at 16:46

You can get all the URLs of a tweet using Twitter Entities. When you make the REST call, make sure you include

&include_entities=true

This will give you a section in the JSON or XML called entities. There will be a child node called urls.

Here's an example of what will be returned.

"text": "Twitter for Mac is now easier and faster, and you can open multiple windows at once http://t.co/0JG5Mcq",

    "entities": {

      "media": [

      ],

      "urls": [

        {

          "url": "http://t.co/0JG5Mcq",

          "display_url": "blog.twitter.com/2011/05/twitte…",

          "expanded_url": "http://blog.twitter.com/2011/05/twitter-for-mac-update.html",

          "indices": [

            84,

            103

          ]

        }

      ],

      "user_mentions": [

      ],

      "hashtags": [

      ]

    }

So, look for entities -> urls to see if a tweet contains a link to an external site.

How to identify url in tweet using R

3 Answers3