MongoDB field order and document position change after update

Question

I am learning MongoDB and I noticed that whenever I do an update on a document the field being updated is pushed to the end of the order, so if I had something like:

db.collection.save({field1: value1, field2: value2, ..., field 10: value10});
db.collection.update({field1: value1}, {$set: {field2: new_value}});

then if you do:

db.collection.find();

it will display:

{ "field1":"value1", ..., "field10":"value10", "field2":"new_value"}

You can see how the field order changes where the updated field is being pushed to the end of the document. In addition, the document itself is being pushed to the end of the collectoin. I know that it's a "schema-less" DB and it may not be a huge problem, but it just doesn't look "pretty" :). Is there a way to do an in-place update without changing the order?

Answer 1

MongoDB allocates space for a new document based on a certain padding factor. If your update increases the size of the document beyond the size originally allocated the document will be moved to the end of the collection. The same concept applies to fields in a document.

Answer 2

FYI, in MongoDB 2.6 updates will preserve field order, with the following exceptions:

1. The _id field is always the first field in the document.
2. Updates that include renaming of field names may result in the reordering of fields in the document.

Answer 3

Both document structure and collection structure in MongoDB based on JSON principles. JSON is a set of key/value pairs (in particular fieldName/fieldValue for document and index/document for collection). From this point of view it doesn't seem that you can rely on order at all.

Answer 4

In the case of the documents if the field size changes, it writes out a new document with the fields sorted by field name. This behavior can be seen with the following statements

Case 1: No change in size of field, so no change in field order

> db.testcol.find()
> db.testcol.save({a:1,c:3,b:2})
> db.testcol.find()
{ "_id" : ObjectId("4d5efc3bec5855af36834f5a"), "a" : 1, "c" : 3, "b" : 2 }
> db.testcol.update({a:1},{$set:{c:22}})
> db.testcol.find()
{ "_id" : ObjectId("4d5efc3bec5855af36834f5a"), "a" : 1, "c" : 22, "b" : 2 }

Case 2: Field size changes and the fields are reodered

> db.testcol.find()
> db.testcol.save({a:1,c:"foo",b:2,d:4})
> db.testcol.find()
{ "_id" : ObjectId("4d5efdceec5855af36834f5e"), "a" : 1, "c" : "foo", "b" : 2, "d" : 4 }
> db.testcol.update({a:1},{$set:{c:"foobar"}})
> db.testcol.find()
{ "_id" : ObjectId("4d5efdceec5855af36834f5e"), "a" : 1, "b" : 2, "c" : "foobar", "d" : 4 }

Is there a particular reason why you do not want the fields reordered? The above was using 1.8.0_rc0 on OS X

Answer 5

I've created a project that creates a custom mongorc.js which sorts the document keys by default for you. It's called Mongo Hacker

Answer 6

In order to have the fields in the order I wanted, I inserted all the documents with the properties in the correct order to another Collection. Then I removed the old Collection and renamed the new Collection to the original name. Obviously backup your data first.

Answer 7

Raman , is right , we can't sort an dictionay , but we can sort the visualization of the dictonary, so we can't sort filed order of an documentation , but I can see it ordered .

For example in perl to_json have option canonical

print to_json( $data, { utf8 => 1, pretty => 1, convert_blessed => 1, canonical => 1 } );

(canonical option) will output JSON objects by sorting their keys. This is adding a comparatively high overhead . (of course, we do more a sorting operation ...)

Answer 8

I had a similar issue to yours where updating a field would push it to the end of the document. The only decent solution I could find was to perform:

1. an aggregation combining $replaceWith and $out stages. The $replaceWith stage handled the rearranging of the fields in all documents within the collection, and the $out stage writes the output of the $replaceWith stage into a new collection.
2. Then you would need to drop the old collection, and...
3. (optionally) rename the new collection.

I'm not sure if you could overwrite the collection on the $out stage by using the same name of the input collection and skip steps 2 & 3.

Otherwise, the commands would look something like this:

> db.oldCollection.aggregate([{$replaceWith: {field1: "$field1", field2: "$field2"... fieldN: "$fieldN"}}, {$out: "newCollection"}])
> db.oldCollection.drop()
> db.newCollection.renameCollection("oldCollection")

The order in which you define the fields in the $replaceWith stage is the order in which they will be outputted. And yes, the values of the fields should be the field names themselves prepended with $.