How to fetch more than 25 post messages

Question

I'm trying to get all post messages using restfb, my code is as follows

public Connection<Post> publicSearchMessages(Date fromDate, Date toDate) {
    Connection<Post> messages = publicFbClient.fetchConnection("search",
            Post.class,
            Parameter.with("q", "Watermelon"),
            Parameter.with("since", fromDate),
            Parameter.with("until", toDate),
            Parameter.with("type", "post"));

    return messages;
}

This only gives latest 25 post messages.

Parameter.with("limit",100 )

If i set limit parameter, it gives 100 messages but i don't want to limit to fetching post messages. So,

Is there anyway I can get a full list of post messages matching the search criteria without setting limit parameter?

Default set is 25 returned for all queries. You will have to use the limit parameter to go beyond that. — ShawnDaGeek, Oct 31 '11 at 00:04
Think you have to deal with `limit` because FB won't let you raise to much load on its machines by unlimiting result size. — micfra, Nov 03 '11 at 14:42

score 5 · Answer 1 · edited Mar 31 '12 at 02:10

5

Maybe you can try using a loop. FB can't get more than 1000 each time, so you can use the loop to get the whole feeds. Use the offset like this:

Parameter.with("limit", 1000));
Parameter.with("offset", offset));

Offset will be a variable and its value will be 1000,2000,3000...

edited Mar 31 '12 at 02:10

Tim M.

53,671
14
120
163

answered Mar 30 '12 at 08:44

Zam

51
1
2

score 2 · Accepted Answer · answered Nov 04 '11 at 07:55

2

There is no way to fetch unlimited results from FB. The default limit is set to 25. As you know, you can change this using the limit parameter. I have not found an upper border for limit searching the web. Maybe, you can set it to a very high amount.

answered Nov 04 '11 at 07:55

micfra

2,780
1
23
34

score 0 · Answer 3 · answered Sep 10 '14 at 08:30

As I tested recently, You don't have to specify anything. Connection class implements Iterable in this way:

fetch 25 results
hasNext check if there is next item to process
if not, it will fetch next page of 25 results

So basically all you need to do is this:

Connection<Post> messages = publicFbClient.fetchConnection("search",
        Post.class,
        Parameter.with("q", "Watermelon"),
        Parameter.with("since", fromDate),
        Parameter.with("until", toDate),
        Parameter.with("type", "post"));

for (List<Post> feedConnectionPage : messages) {
        for (Post post : myFeedConnectionPage) {
                 // do stuff with post
        }
}

If you want to have some kind of method that returns results I would be very careful, because you can be returning thousands of results and crawling through them may take some time (from seconds to minutes or even hours) and result object array is going to be really big. Better idea is to use some asynchronous call and check out results of method periodically.

Though it seems that parameter "since" is ignored. Posts are fetched from newest to oldest and I think that it somehow leave out this parameter when doing paging.

Hope I made it more clear for you :)

score 0 · Answer 4 · answered Mar 18 '15 at 16:10

We have an Iterator object in Post. So we can do it like this:

Connection<Post> messages = publicFbClient.fetchConnection(...) ;
someMethodUsingPage(messages);
    while (messages.hasNext()) {
        messages = facebookClient.fetchConnectionPage(messages.getNextPageUrl(), Post.class);
        someMethodUsingPage(messages);
    }

Then in each messages we'll have next 25 messages.

How to fetch more than 25 post messages

4 Answers4

Linked