Mongodb aggregate error

Question

this is the command and the error generate:

db.tweets.aggregate(
    {$project:{'entities.hashtags.text':1}},
    {$unwind:'$entities.hashtags'},
    {$group:{_id:'$entities.hashtags.text'}})

{
    "errmsg" : "exception: aggregation result exceeds maximum document size (16MB)",
    "code" : 16389,
    "ok" : 0
}

i would want to do a follow query:

group by entities.hashtags.text and count the number of document that contains that hashtags for every hashtags taht exist.

this is a part of document:

...

entities: {

   media: [

         ...

    ],

    urls: [],

    hashtags: [

        {

            text: "makeuploos",

            indices: [

                54,

                65

            ]

        },

        {

            text: "onbewerkt",

            indices: [

                66,

                76

            ]

        },

        {

            text: "hoer",

            indices: [

                77,

                82

            ]

        }

    ],

...

how can i do this??

Could you include a couple sample "entities" documents? In particular the array that $unwind is applied to is not clear to me. — mjhm, Dec 28 '12 at 17:22
same as http://stackoverflow.com/questions/14047572/mongodb-using-group-by-aggregate-framework-to-get-unique-strings/? — Miguel Cartagena, Dec 28 '12 at 18:31

score 0 · Answer 1 · answered Dec 29 '12 at 12:21

0

Add a $where somewhere after unwinding and try match only relevant data. You just have to many different hashtags, and they do not fit in 16MB limit.

answered Dec 29 '12 at 12:21

madhead

31,729
16
153
201

score 0 · Answer 2 · answered Nov 27 '14 at 06:07

From MongoDB v.2.6 you can use option allowDiskUse. For instance:

  db.tweets.aggregate(
    [
      {$project:{'entities.hashtags.text':1}},
      {$unwind:'$entities.hashtags'},
      {$group:{_id:'$entities.hashtags.text'}}
    ],
    {
      allowDiskUse: true
    }
  )

This will enable writing data to temporary files. You can find more info here: http://docs.mongodb.org/manual/core/aggregation-pipeline-limits/#agg-memory-restrictions

Mongodb aggregate error

2 Answers2