Filter urls from text using Java

Question

I have text stored in a database and I want to filtered the urls that the text contain. How is it possible to filter the urls from text using Java code. For example I have the following text inside my db "The dress-a-likes! Try to look normal and this is what happens. @ Bar Louie http://t.co/sNVcoqT0Bc". How can I filtered the link http://t.co/sNVcoqT0Bc.

    Pattern p = Pattern.compile("http://.*|www\\..*");
    DBCursor cursor = coll.find(query);
    while(cursor.hasNext()) {
         System.out.println(cursor.next().get("text"));

         Matcher m = p.matcher("http://...");
}

How can I filtered the cursor.next().get("text") with the matcher. Cursor... is an object while matcher waiting for a String. How can I convert that object to String?

Search for known URI schemes and then read everything til the next whitespace. — Tom, Dec 11 '14 at 08:52
Will you have the link at the end of the text only ? If yes you can use indexOf. If no you can do what @Tom has suggested above — Sariq Shaikh, Dec 11 '14 at 08:52
How using like `str.substring(str.indexOf("http"));` if the url is ever at the end of the main string? — Wundwin Born, Dec 11 '14 at 08:53
No this is just an example. Basically the text I have stored in my db is tweets from twitter. — Jose Ramon, Dec 11 '14 at 08:53
Is tweets from twitter, I want to filter the links from all the tweets I have stored in my db. — Jose Ramon, Dec 11 '14 at 08:56
Check this http://stackoverflow.com/questions/5713558/detect-and-extract-url-from-a-string — Sariq Shaikh, Dec 11 '14 at 08:57
Yes this is exactly what I tried. However, how can I be sure that I ve got all the urls using http? — Jose Ramon, Dec 11 '14 at 08:59

score 4 · Accepted Answer · answered Dec 11 '14 at 08:57

I would try to locate where the "http://" and then take te whole string until the end.

Use: int indexOf(String str)

If there is the posibility of having something more after the URL, then locate the space using another indexOf().

Now use: indexOf(String str, int fromIndex) where fromIndex should be the index finded before.

Make a substring from one of the index until the other.

Use: string substring(int beginIndex, int endIndex)

score 3 · Answer 2 · answered Dec 11 '14 at 09:00

3

Try using ANTLR to parse your file. Create a simple grammar that extracts the links alone. the links end when there is a space character " ". This will parse your whole file and returns all the URL (if there are more than one).

answered Dec 11 '14 at 09:00

Narayana

363
3
14

Filter urls from text using Java

2 Answers2