100 TB of data on Mongo DB? Possible?

Question

What kind of an architecture is needed to store 100 TB data and query it with aggregation? How many nodes? Disk size per node? What can the best practice be?

Every day 240GB will be written but the size will remain same because the same amount data will be deleted.

Or any different thoughts about storing the data and fast group queries?

Yes, it is, the related question is out of date since there have been bigger scenarios on the user group since. — Sammaye, Jan 22 '13 at 08:07
You tagged this with the vertica tag. Do you want some sort of information about that as well? — geoffrobinson, Jan 23 '13 at 15:37

score 3 · Answer 1 · edited May 23 '17 at 12:18

3

Kindly refer to related question,

MongoDB limit storage size?

Quoting from the the top answer:

The "production deployments" page on MongoDB's site may be of interest to you. Lots of presentations listed with infrastructure information. For example:

http://blog.wordnik.com/12-months-with-mongodb says they're storing 3 TB per node.

edited May 23 '17 at 12:18

Community

1
1

answered Jan 22 '13 at 07:52

Samuel Liew

76,741
107
159
260

score 3 · Answer 2 · answered Jan 24 '13 at 22:36

3

I highly recommend HBase.

Facebook uses it for its Messages service, which in Nov 2010 was handling 15 billion messages a day.

We tested MongoDB for a large data set but ended up going with HBase and have been happily using it for months now.

answered Jan 24 '13 at 22:36

Suman

9,221
5
49
62

1

how did you handle infrastructure management? We're a small startup and don't have resources yet to do it at 100% – noli Jun 03 '14 at 20:23
2

Sorry, maybe I'm not understanding - what do you mean by infrastructure management? You mean managing the Hadoop/HBase cluster? We used Amazon Elastic MapReduce. – Suman Jun 03 '14 at 20:49

100 TB of data on Mongo DB? Possible?

2 Answers2