3

I have a collection with an field named date (also indexed). It holds the values yyyymmdd (example: 20140731, 20140730, 20140729...)

The documents are stored in descending order based on date. So the collection has 20140731 as the first document.

When I use the find command with filters {$gte : 20140720, $lte : 20140731}, mongodb returns back the query in ascending order of date field.

I know I can use sort, but how do I get mongodb to return results based on the order it was created?

Thanks!

zhulien
  • 5,145
  • 3
  • 22
  • 36
LAX_DEV
  • 2,111
  • 3
  • 19
  • 29
  • 1
    Use [`natural` ordering](http://docs.mongodb.org/manual/reference/operator/meta/natural/): `.sort( { $natural: 1 } )`? – alecxe Aug 02 '14 at 02:15
  • @alexce Natural ordering reflects order on disk, not insertion order (especially over time) unless you are inserting into a capped collection. See also: [What does Mongo sort on when no sort order is specified?](http://stackoverflow.com/questions/11599069/what-does-mongo-sort-on-when-no-sort-order-is-specified). – Stennie Aug 02 '14 at 07:01

1 Answers1

15

Documents are stored in natural order

The documents are stored in descending order based on date. So the collection has 20140731 as the first document.

Unless you are using a capped collection, there is no guarantee for the ordering of documents on disk (also referred to as natural order).

Document deletions and moves (when a document outgrows its allocated record space) create space on the free list which will be reused.

Here's a quick example which should demonstrate this in the mongo shell:

// Start with an empty database & collection
use demodb; db.dropDatabase(); db.order.drop()

// Add some test data
for (i=0; i<1000; i++) {
    db.order.insert({'i': i})
}

// Looks like insertion order! (0..9)
db.order.find({}).limit(10);

// Pause 5s for effect :)
sleep(5000);

// Remove half the entries
db.order.remove({ i: { $lt: 500 }})

// Re-add the missing entries
for (i=0; i<500; i++) {
    db.order.insert({'i': i})
}

// Not the entries you expected .. space from deleted records was reused
db.order.find({}).limit(10)

// Clean up demodb
db.dropDatabase()

Order of results

When I use the find command with filters {$gte : 20140720, $lte : 20140731}, mongodb returns back the query in ascending order of "date" field.

If an index is used for a query, the documents are returned in the order they are found in the index. You should take advantage of this when constructing your indexes for common queries (see: Use Indexes to Sort Query Results).

FYI, a simple index (eg. on {date:1}) can be used to return the results sorted in either ascending or descending order.

Sort by ObjectID

If you are using MongoDB's default ObjectIDs for _id, you can sort by { _id: 1 } to approximate insertion order since the first 4 bytes of the ObjectID incorporate a timestamp. If you wanted to use this for sorting a query based on date and approximate insertion order you would ensure an index on {date:1, _id:1}.

Note that ObjectIDs are typically generated by the client driver, so if you have clock drift on your app servers (or the _id is created some time before the document is inserted) the ObjectIDs may not strictly reflect "insertion order" as seen by the server. If accuracy of insertion order is highly important, it is generally possible to generate the _id on the server-side (approach varies depending on the driver).

zhulien
  • 5,145
  • 3
  • 22
  • 36
Stennie
  • 63,885
  • 14
  • 149
  • 175
  • how is order maintained for documents inserted in bulk? like if i insert 5 records at once and has same timestamp – Akshay Naik Dec 04 '19 at 13:49
  • 1
    @AkshayNaik Order of documents on disk for bulk inserts is not treated differently from individual inserts. Bulk write operations can be processed as [*ordered* or *unordered*](https://docs.mongodb.com/manual/core/bulk-write-operations/#ordered-vs-unordered-operations), but that only determines whether those operations are executed in serial or parallel. The storage engine manages the order of documents on disk, which is undefined for the default WiredTiger storage engine as well as the older (and now deprecated) MMAPv1 storage engine. – Stennie Dec 05 '19 at 00:00
  • In my case I have over millions of documents in single collection two day back I restarted the server and the order of documents in all collections got reshuffled, earlier it was in insertion order and now the order is random. I am looking for reason under what circumstances it happens. – Akshay Naik Dec 05 '19 at 06:35
  • 1
    @AkshayNaik With WiredTiger the representation of documents in memory is different from the on-disk format. If you want predictable ordering of results you need to specify a sort order on a unique field. – Stennie Dec 05 '19 at 21:10
  • you are right adding a sort({'_id' :1}) gave me original order. – Akshay Naik Dec 06 '19 at 05:45