1

I see below log message in my solr logs

webapp=/solr path=/admin/ping params={} hits=1421599 status=0 QTime=26

o.a.s.c.ShardLeaderElectionContextBase make sure parent is created 
o.a.s.c.ZkController Core needs to recover:

what does hits in this log indicate ?, after this log , I see the cluster went down and failed leader election process. trying to understand is it high volume of reads and writes that took down the cluster ?

arjunsv3691
  • 791
  • 6
  • 19
  • 2
    The ping request just confirms that the core is available and responding. The `hits` is the total number of documents in the core, and the `QTime` is low - meaning that this probably didn't put any pressure on any nodes at all. Does this happen every time you issue a ping request? I might suspect that this is a red herring and that the error lies somewhere else (have you confirmed that your JVM isn't running out of memory?) Which version of Solr? – MatsLindh Sep 01 '21 at 10:03
  • Thanks @MatsLindh , I am running solr 7.7, this issue is occurring intermittently, solr nodes keep running as usual and all of a sudden I see huge cpu spikes and nodes unable to choose leader and eventually cluster goes down. I tried getting logs around the time I observed CPU spike and only these ping logs and couple of read and update queries were seen at that time. Also my solr-promotheus-exporter is a older version it's failing to parse metrics values and keeps throwing error. so can that have any impact ? – arjunsv3691 Sep 01 '21 at 10:20
  • Huge CPU spikes can be corelated to garbage collection or something similar; have you tried enabling verbose GC logging for your Solr nodes to see if something happens around that time? Since the entry gets logged, I would guess that it isn't the cause - since logging has to happen after everything else has been processed. Upgrading Solr might be something you'd want to look into; I wouldn't think that the exported could be the cause (as I'm guessing it only reads from the Solr API and doesn't integrate deeply into Solr?) – MatsLindh Sep 01 '21 at 11:01
  • @MatsLindh I a using 9GB for solr heap. from the docs it's recommended number is 2Gb can this have any impact on CPU hikes, also if I don't provide any JVM OPTS for JVM memory, but the VM has 16 GB of memory. – arjunsv3691 Sep 01 '21 at 17:33

1 Answers1

0

The hits in the Solr logs is the same value of numFound returned by a Solr query. In other words, hits represents the number of documents that matches the query.

In your case the hits logged in response of the handler /admin/ping represents the number of documents in the Solr core/replica.

freedev
  • 25,946
  • 8
  • 108
  • 125