1

I have something of an abstract question regarding managing live feeds/polling on web sites.

I am creating a web app (built on Java/Spring/Hibernate) and on the user's home page I want a live feed of the latest activity from all the members of there team, and I am trying to work out the best way to handle this query on the server side.

The brute force way would be to load the current users list of team mates, and then iterate through each of his team mates, loading their latest conversations/file uploads/etc, and then merging all this activity in to a single list sorted by timestamp and returning that (lets say for sake of example that we just return the top 10 latest activity for the feed).

However, that seems very un-performant, especially as this operation would need to be done regularly (depending on the polling interval).

I have also considered making all the potential activities (conversations/status updates/uploads) as extending an Activity class and then just having a direct SQL/JPQL query in a DAO that selects all the latest activity from a set of users to be returned, but concerned that might bypass the caching and continued access to the DB would also reduce performance.

Has anyone handled this type of problem before? any one know what a good approach is?

Thanks!

rhinds
  • 9,976
  • 13
  • 68
  • 111
  • Although I've never had to implement something like this, I guess pubsubhubbub solution might help you: http://code.google.com/p/pubsubhubbub/ – Lucas de Oliveira May 11 '11 at 14:42
  • Thanks Lucas, I have had a quick look but this doesnt quite look like what I need - maybe I didnt describe my question very well, but I am going to be creating a feed from by data model (in Hibernate) and want to find an efficient way of processing the Hibernate objects regularly to create the feed. – rhinds May 20 '11 at 10:04

1 Answers1

0

This is an old one now, but here is what i did for this:

  1. All tasks that should appear on a live wall extend Activity (this was already the case)
  2. Created a new Notification object, the Notification had a link to the underlying Activity and a link to a user (who was being notified).
  3. Created a pre-persist hook for Activity that created a Notification object for the Activity being persisted - it did this for every user that was interested (all users following the user that was persisting the Activity)

For the moment, Notifications are persisted/retrieved to the DB - possibly not scalable to very high volumes, but the approach I think supports moving to a Queue based system (such as LinkedIn's Kafka queue library which is designed exactly for this purpose). As it is per-user, it also provides the option to have a read/unread notification flag for significant notifications.

rhinds
  • 9,976
  • 13
  • 68
  • 111