How do i start a cassandra node which is down and unpingable?

Question

I have a 3 node cluster and one of them is down and i'm unable to ping it i could see below error in the server.log from one of the up node

ERROR [ScheduledTasks:1] 2019-10-01 05:26:16,658 ResourceWatcher.java:64 - Timed run of class org.apache.cassandra.locator.PropertyFileSnitch$1 failed.
org.apache.cassandra.exceptions.ConfigurationException: unable to locate cassandra-topology.properties
        at org.apache.cassandra.utils.FBUtilities.resourceToFile(FBUtilities.java:303) ~[apache-cassandra-3.11.4.jar:3.11.4]
        at org.apache.cassandra.utils.ResourceWatcher$WatchedResource.run(ResourceWatcher.java:53) ~[apache-cassandra-3.11.4.jar:3.11.4]
        at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) [apache-cassandra-3.11.4.jar:3.11.4]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_91]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_91]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_91]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_91]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
        at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81) [apache-cassandra-3.11.4.jar:3.11.4]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]

I could see below info when i did a tail:

INFO  [IndexSummaryManager:1] 2020-05-21 02:41:16,017 IndexSummaryRedistribution.java:77 - Redistributing index summaries
INFO  [pool-1-thread-3] 2020-05-21 02:45:34,023 NoSpamLogger.java:91 - Maximum memory usage reached (507510784), cannot allocate chunk of 1048576
INFO  [pool-1-thread-3] 2020-05-21 03:00:34,039 NoSpamLogger.java:91 - Maximum memory usage reached (507510784), cannot allocate chunk of 1048576
INFO  [pool-1-thread-3] 2020-05-21 03:15:34,062 NoSpamLogger.java:91 - Maximum memory usage reached (507510784), cannot allocate chunk of 1048576

@Ali Ahmad : the latest logs shows only the info which i posted above — Rajesh Damam, May 21 '20 at 07:43
i want to bring up the node which is down, i couldn't ping or ssh it, Is there any way i can bring up that Node. — Rajesh Damam, May 21 '20 at 07:45
Kindly read this may resolve your issue https://stackoverflow.com/questions/51591253/cassandra-nospamlogger-log-maximum-memory-usage-reached — Ali Ahmad, May 21 '20 at 13:42

How do i start a cassandra node which is down and unpingable?

0 Answers0