Mongodb background indexes - are they still background once created?

Question

When creating an index in mongodb, you can specify the background: true flag, which causes the index creation to be non-blocking. This is great in production since you don't want the whole database locked up while creating an index that you clearly didn't critically need before (since you didn't have it).

Reading the docs, it seems like this flag only determines how the index is created, and once it's done being built the index acts exactly like a normal index. Which is what I'd want -- I wouldn't want the index to get out of sync with docs because it's being updated in the background, although I can imagine a database that does this.

I'm asking here because the getIndexes command shows that the index is still marked as background even after it's created. Is this just a reminder about how it was created? Or do background indexes behave differently after being created? Maybe some subtlety with replication?

score 27 · Accepted Answer · edited Mar 27 '13 at 18:03

Once created they act just like regular indexes. They persist in getIndexes just as a reminder, similar to how unique, sparse and so on do.

But it also has the other meaning. Just because foreground indexes block all writers, in this case you won't be able to perform db.testCollection.getIndexes() until all indexes have been created. Meanwhile when you create background index, then you can call db.testCollection.getIndexes() and you will see, that index seems to be already created.

But in this case we can't be sure whether indexes have been actually created or not. In this case you need to call db.currentOp() and if you see something like

{
  "inprog": [
    {
      "opid": 2001060,
      "active": true,
      "secs_running": 1,
      "op": "insert",
      "ns": "test.system.indexes",
      "insert": {
        "v": 1,
        "key": {
          "a": 1
        },
        "ns": "test.testCollection",
        "name": "a_1",
        "background": 1
      },
          ....
      "msg": "bg index build Background Index Build Progress: 368640/1000000 36%",
      "progress": {
        "done": 368640,
        "total": 1000000
      }
      ...
    }
  ]
}

then it means, that creation of background indexes is still in progress, and also you can see some information about the process.

For example you may do some rough calculations: 368640 out of 1000000 takes 1 seconds (+1 second as possible offset), therefore everything should take 3-6 seconds (eventually it took 4.8 s).

Obviously if you can't see such operation in progress, then indexes are already created.

Note: if you have many concurrent operations, then you may specify searсh argument for db.currentOp(), f.e.

db.currentOp({"insert.background":1})

hmm both unqiue and spare, despite being created as such do persist as such also past creation, so I am not sure I get the first line. — Sammaye, Mar 24 '13 at 14:43
I'm just trying to say, that `sparse`, `unique` are just identifiers, which you may see if you call `db.collection.getIndexes()`. F.e. if you create index as `sparse`, then you'll see `{sparse : true}` in its description. And there is the same behavior for `background`, `dropDups` and other arguments. In other words, everything what is passed as second argument in `ensureIndex` comes to index's description. — n1ckolas, Mar 24 '13 at 15:28
I updated the first paragraph to reflect this comment discussion. — Leopd, Mar 27 '13 at 18:15
Note that using dropDups:true may cause the percentage complete to be calculated incorrectly as the total number of documents decreases: `"msg" : "bg index build Background Index Build Progress: 67364294/27257585 247%"` — Dave Walker, Apr 11 '14 at 15:13
In 3.0.12 I used this query to filter currentOp: db.currentOp({"query.indexes.background":true}) — James Watkins, Jul 20 '16 at 23:55

Zameer Ansari · Answer 2 · 2016-09-04T12:01:03.680

There are multiple indexing options - foreground (default) & background. Foreground is relatively fast and it blocks all writers and readers. Other databases we can still get to. This is supposed to be not done in a production environment.

Background index creation is a bit slower and they don't block readers and writers. With MongoDB 2.4 and later, you can create multiple background indexes in parallel even on the same database.

Beginning in MongoDB 2.6, creating an index in the background on the primary will cause the indexes to be created in the background on secondaries, as well. The secondaries will begin index creation when the primary completes building its index.

There is another way to create index very efficiently in a production system. That is to create an index on a different server that is being used to serve most queries. Say, in a replica set of multiple database servers working in tandem, one can be taken out and the requests can be routed to the available ones. The foreground index creation can be done on the separated server. After the creation is done successfully, it can be brought back to the cluster.

Mongodb background indexes - are they still background once created?

2 Answers2