How the 10 million questions counters on stackoverflow are made?

Question

Stackoverflow is celebrating 10 Million Questions. Congratulations!

Providing us this link: https://stackoverflow.com/10m

There are 3 counters displayed, and the number of each counter is increasing fast and not static.

However, I don't see any AJAX requests for displaying the recent result of each counter.

I got such a counter in our forum as well, but to get the number of recent posts, I do an AJAX request every 3 seconds, running the query SELECT MAX(id) AS total_posts FROM forumposts to display.

I know, this is not the best solution we have, and it will be not correct anymore if a post is deleted. Using the SELECT COUNT(id) command is too slow, since we have also more than 10 million posts.

So, how Stackoverflow is displaying the increase of their counters without any requests? It seems to be a nicer solution for me and I'd like to use it for our forum as well then.

There's a few related questions on meta, like this one http://meta.stackoverflow.com/q/302925/1301076 — rjdown, Aug 31 '15 at 12:32
It increases depending on average questions submissions no need to ajax request, you can figure it out by monitoring this page http://stackoverflow.com/unanswered — Radi, Aug 31 '15 at 12:33
I'd say it's random, based on previous activity on the site. — Michael Kunst, Aug 31 '15 at 12:34
Yes, my question is diffrent to the meta discussion. Looking for a solution and actually, I can imagine it's some kind of average. Since it's not increasing statically, I was interested, seems to be more behind... — lickmycode, Aug 31 '15 at 12:36

score 7 · Answer 1 · edited May 23 '17 at 12:29

7

It's using websockets. You can see it from the network tab. wss://qa.sockets.stackexchange.com/ From the request, it does look like it gets a total count everytime.

SQL count being slow A poor man's way of improving performance would be to track statistics yourself by creating a table thats keeps track of the row count. You'd be burdened with making sure that you update that table every time you delete or add new records. Before you do that, do some research on indexes and the following questions

edited May 23 '17 at 12:29

Community

1
1

answered Aug 31 '15 at 12:35

Ruan Mendes

90,375
31
153
217

I saw this. Reading now about websockets. It's the communication protocol between server and client. And what do you think is the background of each counter? Is Stackoverflow really counting millions/billions of questions, edits, votes and more from database every millisecond to send the correct number? – lickmycode Aug 31 '15 at 12:44
I assume these stats are generated based on the average of the past years. Requesting for all these stats from the db every second or two would result in a considerable server resource consumption without any big gain. – Rehmat Aug 31 '15 at 12:54
@lickmycode They could also not rely on the db to count all the records, they may have a table where they keep just stats. But we can't really know... – Ruan Mendes Aug 31 '15 at 12:55
A statistic table would make sense, but even with this, how this dynamic (non-static) increasing numbers are coming from? Since only SO can answer this, how would you do such counters (non-static)? – lickmycode Aug 31 '15 at 13:22
@lickmycode I don't understand what you are asking.... You have to update your statistics table every time you add/delete a record – Ruan Mendes Aug 31 '15 at 13:25

How the 10 million questions counters on stackoverflow are made?

1 Answers1