2

I'd like to start over with errbit - there are millions of records in our mongodb database and we hadn't been cleaning them up. I'd like to start over, but I don't want to lose my user accounts.

I've tried to run these routines (https://mensfeld.pl/2015/01/making-errbit-work-faster-by-keeping-it-clean-and-tidy/):

bundle exec rake errbit:clear_resolved

desc 'Resolves problems that didnt occur for 2 weeks'
task :cleanup => :environment do
  offset = 2.weeks.ago
  Problem.where(:updated_at.lt => offset).map(&:resolve!)
  Notice.where(:updated_at.lt => offset).destroy_all
end

but the second one (deleting problems and notices over 2 weeks old), just seems to run forever. Querying problems and notices collections via mongo shell doesn't seem to show any being deleted... we're using errbit V 0.7.0-dev and mongodb version 3.2.22.

Vasfed
  • 18,013
  • 10
  • 47
  • 53
mark amos
  • 321
  • 2
  • 5
  • Seems like a good usecase for [TTL indices](https://docs.mongodb.com/manual/core/index-ttl/) if it is a single collection. Unfortunately I am not familiar with errbit to provide proper answer. The benefit of the index is it triggers db side so no data exchange, round trips etc. – Alex Blex Jul 29 '19 at 16:14

2 Answers2

1

Fastest way would be to get a mongo console and drop most of the collections. I'd say stop your errbit server, get a mongo console, connect to the db you use and run:

> db.errs.drop()
true
> db.problems.drop()
true
> db.backtraces.drop()
true
> db.notices.drop()
true
> db.comments.drop()
Stephen Crosby
  • 1,157
  • 7
  • 19
1
Problem.where(:updated_at.lt => 2.months.ago).destroy_all

runs too long because of N+1 problem with recursive deletion of Err, Notice and Comment, also mongoid does not support nested eager loading, so only way to delete faster - is to manually take these ids and delete directly, without callbacks:

problem_ids = Problem.where(:updated_at.lt => 2.months.ago).pluck(:id)
err_ids = Err.where(problem_id: {:$in => problem_ids}).pluck(:id)

Notice.where(err_id:{:$in => err_ids}).delete_all
Err.where(id:{:$in => err_ids}).delete_all
Comment.where(problem_id: {:$in => problem_ids}).delete_all
Problem.where(id: {:$in => problem_ids}).delete_all
Vasfed
  • 18,013
  • 10
  • 47
  • 53