Reliably stopping an unresponsive thread

Question

I'm wondering how to stop an unresponsive thread in Java, such that it's really dead.

First of all, I'm well aware of Thread.stop() being deprecated and why it should not be used; there are already many excellent answers on this topic, cf. [1][2]. So, the question more precisely is, whether it's actually technically possibly to kill a thread which code is not controlled by us but possibly hostile and not responding to interrupts.

In the simplest case, a hostile thread would be running while(true);, but it could as well be using up memory or other system resources to do more damage. Calling interrupt() on that thread is obviously ineffective. What about calling stop() instead?

I have run this in a debugger and, in fact, the thread really disappears. But is this approach reliable? The hostile thread could be prepared for this case; think of try{run();}catch(ThreadDeath t){run();} where it catches the ThreadDeath that is produced when we call stop() and recursively calls itself again.

As an outside observer, we cannot see what is going on; Thread.stop() always runs silently. Worst of all, the usual diagnostics won't work anymore (tried this while debugging on Corretto 1.8.0_275 Windows x64): Thread.getState() always returns RUNNABLE regardless of success in killing the thread, same goes for Thread.isAlive() (always true).

If you execute untrusted code in the thread there may be more severe problems than just DOS. — Henry, Dec 30 '20 at 07:39
Most if not all implementations of the jvm use system level threads. Is it an option to let the OS kill the thread? — Felix, Jan 28 '21 at 20:14
No- if you were to "pull the rug from under the JVM's feet" by simply killing the thread, then you would potentially leave the Java heap and other state such as locks in an unsafe state. — Neil Coffey, Jan 28 '21 at 21:26
Here are some pointers: https://stackoverflow.com/q/502218/829571 — assylias, Jan 29 '21 at 08:31
Interesting and good answers, in particular on thread-local Security Managers. However, the part on killing the thread is still vague. — pxcv7r, Jan 30 '21 at 08:54
According to my experiments, `isAlive()` *always* returns true if the thread did not terminate normally (by itself or by interruption), as written above. Thus, that loop would just loop forever. — pxcv7r, Jan 31 '21 at 17:44
@matt, I meant 'outside the thread', but inside the same JVM. I assume that we started the thread, but do not know what it executes exactly. — pxcv7r, Jan 31 '21 at 17:51

score 3 · Accepted Answer · answered Dec 30 '20 at 07:58

It may not be possible, at least not reliably in every scenario.

IF I understand the mechanism correctly (and there is some uncertainty there), if the code executes in such a way that there are no safepoints during the execution (for example in counted loops), it is not possible for the JVM to signal to the thread that it should stop (the thread never polls for an interrupt).

In such a scenario, you need to kill the JVM process, rather than the thread.

Some extra reading:

How to get Java stacks when JVM can't reach a safepoint

Counted loops

score 2 · Answer 2 · answered Jan 30 '21 at 23:18

In a nutshell, there's no 100% reliable way to stop a Thread the way you'd like it.

Why?

This is an explanation for others who don't know why, anyone who knows the issue can skip this.

The way how threads are intended to be terminated forcefully is with the interruption state of the Thread. A Thread should be terminated with its interrupt() method is called which sets a boolean flag to true.

When the interruption flag is set to true, the Thread should terminate itself with virtually no delay.

Anyway the Thread can choose to simply ignore this and keep on running.

This is when the stop() method can be called that forces the Thread to terminate. The problem is that this method messes up concurrency, can damage objects and the program can be corrupted without a warning for the user. See Why the stop() method is deprecated?

At the end I could think of two possible ways, one is basically your way, the other one is safer but more complicated.

As an example, a hostile third party .jar which contains a Thread that refuses to terminate can cause these problems.

Quick & Dirty

This solution isn't completely safe but based on the usage this may be acceptable unless you really like security.

Try to first to call the interrupt() method on the Thread and give it a bit time to terminate.

If the Thread doesn't respond, you can either:

terminate the program and warn the user to not run that Thread again.
stop() the thread and hope for the best.

Complicated & Safe

The safest solution I can think of is creating a whole new process to run the Thread in. If the Thread doesn't want to terminate after interrupt(), you can just end the process with System.exit(-1) and let the OS handle it.

You need Inter Process Communication to communicate with the other process and that makes it a lot more complicated but also safer.

import java.util.Arrays;
public class BrokenThreads{
    static boolean[] v = { true };
    public static void call(){
            try{
                while(true){
                    Thread.sleep(200);
                }
            
            } catch ( Throwable td){
                System.out.println("restarting");
                call();
            }   
    }
    public static void main(String[] args) throws Exception{
        
        Thread a = new Thread( BrokenThreads::call);
        
        a.start();
        Thread.sleep(500);
        System.out.println( Arrays.toString( a.getStackTrace() ) );
        while(v[0]){
            a.stop();
            System.out.println(a.getStackTrace().length);
            v[0] = a.isAlive();
        }
        System.out.println("finished normally");
        System.out.println( Arrays.toString( a.getStackTrace() ) );
    }
}

Note that "getStackTrace" takes time, and you can see the stacktrace accumulate as recursive calls are made, until two stops happen quick enough to end the thread.

This uses two techniques to see if the thread has stopped. isAlive and the depth of the stack trace.

score -1 · Answer 4 · answered Jan 31 '21 at 12:22

I think the question describes a scenario that is the reason why Thread.stop() is deprecated since ages now, but was not yet removed … just to have a 'last resort option', to be used only when being really desperate and being aware of all the negative impact.

But that call to Thread.stop() must be build into the code somehow, same as any alternative one may think about – so why not just fix the code for the thread? Or, if that is not possible because that code comes with a third party library without source code, why not replacing that library instead?

Ok, during testing, your own code may go wild, and you need an emergency break – for that, Thread.stop() is still good enough if you do not want to kill the whole JVM (what would be the better option in most of the cases). But again, you have to build this into the code before you start the test …

But in production, there should never be a thread that does not stop when receiving an interrupt. So there should be no need for a replacement of Thread.stop().

score -3 · Answer 5 · edited Jan 28 '21 at 20:10

-3

This can potentially open a can of worms like memory access violations which will kill the JVM itelf.

What you could do is isolate the thread, running .finalize() on it, then forcing the JVM to run GC operations such as Runtime.gc(), System.runFinalization() while forcing interruptions on that particular thread in order to bypass it's resurrection behavior.

I think .finalize() is effectively deprecated since java11 or maybe sooner, so it probably won't help you much.

If you really want to secure your runtime during it's operational cycles, your best bet would be to find a way to essentially map out your configuration before you start it, and have monitoring tools set up which cross-check against that map and monitor the integrity of your runtime while looking for injected classes and/or threads. ... this is assuming of course, you're attempting to guard against "virus-like" attacks in your jvm ... which is not exactly unheard of but still pretty rare.

If you're running some code off the internet, you could simply solve the issue with a call hierarchy inspection and figure out what spawns that problematic thread.

Note: If that invasive thread is calling native dll code which is looping back into it's caller, then your JVM will crash if you mark sections of it's address space as garbage collected.

edited Jan 28 '21 at 20:10

pxcv7r

478
5
14

answered Jan 27 '21 at 18:05

EverNight

964
7
16

I should probably point out the fact that I'm talking out of experience with this kind of stuff from the point of view of someone who experimented with finalization and garbage collection for purposes other that killing particular threads. – EverNight Jan 27 '21 at 18:07
The question is not about how to monitor/prevent changes to memory. It's about terminating a thread. – pxcv7r Jan 28 '21 at 09:22
What do you mean by "isolate the thread"? This may be getting more close to the point. How would calling GC have an effect on the thread runtime? It may remove references from the heap (at most). To force interruptions is precisely what the question is about; how would you do that? – pxcv7r Jan 28 '21 at 09:39
Right, so by "isolating the thread" i mean finding a way for your JVM to detect things inside it which shouldn't be there... like that thread for example. If you find a way of doing this without "monitoring" ... then be my guest. The second part is to effectively destroy the invasive effect. i.e. something that runs `try{run();}catch(ThreadDeath t){run();}` . Since all non-primitive types have to extend the Object class which has the .finalize() method built in, you can take advantage of it using reflection to invoke it periodically with your own "doctor" thread. – EverNight Jan 30 '21 at 04:07
Garbage collection works by going through a number of steps. 1. is marking a particular address space as "to be GC'd" 2. eventually running the actual collection process. These two steps are relatively disconnected for a number of reasons i won't get into, but the basic idea behind forcing finalization and garbage collection is to essentially counteract that idiotic 'try{run();}catch(ThreadDeath t){run();}' implementation. – EverNight Jan 30 '21 at 04:10
say that you implement something that detects the existence of that invasive thread. Now you need to deal with it. As you claim that interrupting the bastard doesn't work because it's self-calling it's own `.run()` directive, the idea would be to essentially get rid of the entire object instead of relying on functionality built into the object itself. So by spawning another thread which calls `.interrupt()`, `sleep()`, `.finalize()` using reflection as well as asking for `Runtime.gc()` immediately after doing so should essentially kill it at a deeper level than Thread.stop(). – EverNight Jan 30 '21 at 04:13
simplly put: spawn another while(true) thread, make it call whatever you can think of to stop the bastard thread from running code in the jvm, while doing so, you're also calling .finalize and Runtime.GC and whatever else you can think of in order to destroy the Object itself. Thread extends Object, get it?. Think about it this way. You can't infect your pc with a virus if you don't run some malware. With the .finalize and gc, you're essentially telling an anti-virus to do it's thing i.e. delete the malware. – EverNight Jan 30 '21 at 04:22
Lastly, because i'm one with this question. You have to essentially force the garbage collection process. This can differ based on a number of factors and most likely, Runtime.GC() won't be enough. (it is on my machine for particular purposes other than dealing with skript kiddie threads). Whether or not you nee to implement System.runFinalization(), Runtime.GC() ... customForceFinalizationMethodNameHere(), etc is up to your skills, dependencies an know-how. – EverNight Jan 30 '21 at 04:37
1

Garbage collection won't kill a running thread. A running thread is a GC root. So finalization is not going to solve anything. – Stephen C Jan 31 '21 at 07:26
If you'd have read 10% of what I wrote you'd have figured out that the whole point was to force that thread in a non-running state, and call GC and finalization while in that state, therefore preventing it's exception handler behavior. – EverNight Feb 02 '21 at 13:10

Reliably stopping an unresponsive thread

5 Answers5

Why?

Quick & Dirty

Complicated & Safe

Related

Linked