Stack Overflow
Stack Overflow

Questions tagged [mongodb-oplog]

A fixed-sized collection that automatically overwrites its oldest entries when it reaches its maximum size which stores an ordered history of logical writes to a MongoDB. Also, the oplog is the basic mechanism enabling replication in MongoDB.

More details : https://docs.mongodb.com/manual/core/replica-set-oplog/

71 questions
13
votes
3 answers

MongoDB: How to get the last updated timestamp of the last updated document in a collection

Is there a simple OR elegant method (or query that I can write) to retrieve the last updated timestamp (of the last updated document) in a collection. I can write a query like this to find the last inserted…
void
  • 2,403
  • 6
  • 28
  • 53
8
votes
1 answer

Why observing oplog takes so much time in meteor / mongo?

I have a MongoLab cluster, which allows me to use Oplog tailing in order to improve performances, availability and redundancy in my Meteor.js app. Problem is : since I've been using it, all my publications take more time to finish. When it only take…
David Panart
  • 656
  • 6
  • 20
6
votes
1 answer

Multiplexers, observeChanges duration and OplogTailingin mongo / meteor

I'm developping an app with Meteor.js. For that, I do have a cluster of mongo instances for oplog tailing. In addition, I'm using Kadira in order to trace my app perfs. Since I've enabled oplog tailing, many of my pub/sub are slower than before.…
David Panart
  • 656
  • 6
  • 20
4
votes
2 answers

MongoDB Change Stream: Can I get value before update/delete?

Change Stream Event on an update operation just return the document that changes to, same as oplog. Can I get the document (or some updated values) before update? MySQL row-based binlog can do this with full binlog_row_image.
auntyellow
  • 2,423
  • 2
  • 20
  • 47
4
votes
1 answer

kafka connector debezium mongodb CDC update/$set message without filter(_id value)

i am trying to setup syncing from mongodb to kudu with debezium mongodb connector. but as debezium doc tell and also i tried by myself and found, there are no filter(_id value) for debezium mongodb CDC update/$set message. { "after": null, …
bronzels
  • 1,283
  • 2
  • 10
  • 16
3
votes
1 answer

MongoDB : applyOps: not authorized on admin to execute command

A Very Good Day, I have the user in the mongodb like below who has the superuser privileges (I confirmed the role setting using show users command) { "_id" : "admin.mongoadmin", "user" : "mongoadmin", "db" : "admin", "roles" : [ …
Harry
  • 3,072
  • 6
  • 43
  • 100
3
votes
0 answers

Migrate MongoDB replica set to new server with minimal downtime

We have a 3 member replica set mongodb running on mLab for a production website. We want to move the database to a new replica set hosted in our own Google Cloud account. My current idea is to do the following steps use dump/restore to copy a…
tjlian666
  • 343
  • 5
  • 17
3
votes
2 answers

Mongo Change Streams not returning all the updates to the collection

I am trying to use Change Streams to lookup/monitor the changes to a mongo collection. cursor = db.collection.watch(full_document='updateLookup') document = next(cursor) print document['documentKey'] While using updateMany to update multiple…
user9447867
  • 31
  • 1
  • 3
3
votes
1 answer

mongodump --oplog and per database restore

I need to be able to restore a single database, even a single collection from a backup. Since mongodump --oplog only applies to a full instance (Replica Set), I made the following procedure to filter only the entries from the db that I want to…
peporro
  • 83
  • 7
2
votes
3 answers

how to convert mongoDB Oplog file into actual query

I want to convert the MongoDB local Oplog file into an actual real query so I can execute that query and get the exact copy database. Is there any package, file, build-in tools, or script for it?
Shatya Kesarwani
  • 53
  • 7
2
votes
1 answer

Read older documents - like `tail --follow` - but for a mongodb collection

I am familiar with the MongoDB oplog and change-streams, like so: const changeStream = d.db(dbName).collection(dbCollName).watch(); changeStream.on('change', next => { log.info('doc:', next.fullDocument); }); but my question is, is there…
user11810894
2
votes
0 answers

Indexing the mongo oplog

Is it wise/recommended/possible to build an index on the oplog.rs collection? The reason I ask is that I believe that it is the cause of our constant alerts regarding scanned object/returned surpassing 1000. Is this possible or does that alert not…
Hemal
  • 1,617
  • 2
  • 12
  • 21
2
votes
1 answer

Querying timestamp field of mongo oplog using java

I am trying to query timestamp field of mongo oplog collection using java. Below is the code. BSONTimestamp timestamp1 = new BSONTimestamp(1499172935, 1); BasicDBObject query1 = new BasicDBObject("ts", new BasicDBObject("$gt", timestamp1)…
Bhavesh Kaushal
  • 41
  • 3
2
votes
1 answer

Cannot print timestamp from mongodb oplog using pymongo

I have the following code: connection = MongoClient('core.mongo.com', 27017) db = connection['admin'] first = db.oplog.rs.find().sort('$natural', pymongo.DESCENDING).limit(-1).next() ts = first['ts'] while True: cursor = db.oplog.find({'ts':…
Animesh Pandey
  • 5,900
  • 13
  • 64
  • 130
1
vote
1 answer

Limit retention in MongoDB Atlas oplog

I have a small dataset stored in MongoDB Atlas with very low write activity. This yields an oplog window of 1.5 years! I have short retention policies, but if I have essentially a point-in-time restore going back over a year then they aren't much…
Michael Haren
  • 105,752
  • 40
  • 168
  • 205
1
2 3 4 5