0

I am looking to write an application that can track a hashtag in twitter, get the id and text, and storing this in a MySQL database. So the application should be able to -

  1. Keep checking to see if anyone has tweeted on a particular hashtag (eg - #example).
  2. Store the details of the tweet.

I have code that can request twitter to search for a hashtag, and returns a list of 20 tweets with the given hashtag.

So now my questions are:

  1. How do I make sure that the same tweet is not pulled up again? (I can always store the tweet ID and check to see if it already exists when adding a new tweet, but not sure if this is the right way to go about it)
  2. I want to do this over a few hours, so will I need to include any kind of refresh code or something that can constantly keep running the search? (Specifically, I want to search for NEW tweets with the hashtag, not existing ones. So anyone who tweets the hashtag AFTER I start tracking, I want those tweets to get saved)

Any more information/code that is needed, please do ask and I can post it here! Bear with me, I am a beginner at this.

Amal Murali
  • 75,622
  • 18
  • 128
  • 150
Sainath Krishnan
  • 2,089
  • 7
  • 28
  • 43

1 Answers1

-1

So, I think you should use regular expressions -> Here

Twitter has the following structure of tweets: 

`<div class="tweet original-tweet js-stream-tweet js-actionable-tweet js-profile-popup-actionable js-original-tweet" data-you-block="false" data-you-follow="false" data-expanded-footer="<div class="js-tweet-details-fixer tweet-details-fixer"> … tabindex="-1">Details</a> </span> </div> </div>" data-user-id="25324805" data-name="~*Beebe♡Bluff*~" data-screen-name="UrBabygurrl" data-item-id="440556385688748032" data-tweet-id="440556385688748032" data-feedback-key="stream_status_440556385688748032"`

Maybe should you filter tweets by data-item-id ? If you want to check for tweets every hour, day, week, month etc. you should use cron :)

ajtamwojtek
  • 763
  • 6
  • 19
  • I am mainly trying to understand how the listening process works :) Do I need to put the get tweets function in some kind of loop statement? Or is it automatically going to get all the tweets by itself? How does it know that a given tweet is new, and wasn't already captured by it previously? – Sainath Krishnan Mar 04 '14 at 03:25
  • 1
    -1 [Don’t parse HTML with regular expressions](http://stackoverflow.com/a/1732454/216074). Also, Twitter offers an API, so there is *really* no need to query the Twitter website. – poke Mar 04 '14 at 14:37
  • Thanks for advice :) I forgot about Twitter API, I'm so stupid :) – ajtamwojtek Mar 04 '14 at 15:19