1

I have a small dataset stored in MongoDB Atlas with very low write activity. This yields an oplog window of 1.5 years! I have short retention policies, but if I have essentially a point-in-time restore going back over a year then they aren't much use.

I see that MongoDB Atlas has a setting for minimum oplog window, but is there a way to set a maximum oplog window?

Michael Haren
  • 105,752
  • 40
  • 168
  • 205
  • You can set the [_"The maximum size of the oplog in megabytes."_](https://www.mongodb.com/docs/manual/reference/command/replSetResizeOplog/). – rickhg12hs Feb 21 '23 at 19:06
  • Thanks @rickhg12hs, unfortunately the minimum you can set the max to is still pretty big (990MB) and this setting is [not available](https://www.mongodb.com/docs/atlas/cluster-additional-settings/#set-oplog-size) in Atlas. – Michael Haren Feb 22 '23 at 18:29
  • No, but out of curiosity, why is it a concern? Atlas is a DBaaS, oplog belongs to the vendor to manage replication between nodes. If you need greater control on the cluster you need to manage it yourself. – Alex Blex Feb 28 '23 at 09:52
  • If you have strict retention rules on data (e.g. for legal reasons), then backups are something you need to be aware of. The oplog is effectively a backup, too. On a busy system with an oplog holding hours or a few days this is often not a problem. But when the oplog stretches into _years_...it's a problem. – Michael Haren Mar 01 '23 at 17:57

1 Answers1

0

The only solution I've come up with is to essentially flush the oplog with nonsense like this:

  1. Use a small loop to insert 100 documents into a new collection `{ value = "hello world" }
  2. Update all the docs a bunch of times with something like value = value + " hello world" until they are large, e.g. 10MB
  3. Update all the docs with a simple update to further spam the oplog with nonsense like value = value + "!"

In about 10 minutes, including figuring out the commands to do all that, I was able to reduce my oplog time from 2 years to 2 hours. But of course it is now growing again...

Obviously this is absurd and not a good path forward .

Michael Haren
  • 105,752
  • 40
  • 168
  • 205