2

I have a javascript widget the loads JSON data from a Java webapp. I want to record impressions, and the ids of the data I return (5 or 10 longs).

I want to be able to process these and give a summary of how many impressions a person got on their widget, and record how many times each piece of data was seen.

I'm guessing it's not a great idea to store it all in my postgres database since it could probably be a lot of data (perhaps tens of impressions per second).

Right now I'm considering just writing it all to a log file and processing it every hour.

Is there a better way? Maybe with some kind of message system, or event system?

John Smith
  • 21
  • 1

2 Answers2

0

Writing to log and processing off-line should be ok.

You can program your logging system to create hourly log files, then process files that are not written-to any more. You can do background processing in separate thread/process at low priority.

Peter Knego
  • 79,991
  • 11
  • 123
  • 154
  • I'm using log4j (through slf4j), how would you recommend separating out the info from the rest of the log file? I'd rather not have hourly log files for everything since normally my logs are low volume and daily works best. – John Smith Feb 17 '11 at 15:20
  • Check this out http://stackoverflow.com/questions/1711423/log4j-rotating-log-files-based-on-time-rather-than-size – Peter Knego Feb 17 '11 at 16:40
0

I'm not sure what you are after, but maybe it's a good solution to do call to an web-analytics (.e.g. Google Analytics) tool from the javascript part of your widget?

You won't have access to your raw data, but it will scale wonderful and you'll have nice graphs.

Notalifeform
  • 194
  • 1
  • 8
  • I'm not familiar with using google analytics. Would I be able to: separate the impressions for each user (different websites would be running the widget, I only want to show them their impressions). Also would I be able to summarize the data (my widget shows links, identified by an ID. Would I be able to give extra data to google analytics with the IDs that were displayed, and access this data?) – John Smith Feb 17 '11 at 16:42
  • hm.. the only way to seperate user-data would be to have a separate analytics account for each user; and you might be able to use event-tracking for displaying the ID's. Google does have API's for accessing reports too. So you might be able to make it work, but I guess it's mybe too restrictive for your requirements. – Notalifeform Mar 15 '11 at 11:56