There's a Java binding for OpenCL, which is the standard API for doing non-graphics GPU computation. If your computer doesn't support OpenCL, you could also use OpenGL to do calculations with a fragment shader, though that's a bit more awkward.
You'll need to rewrite your calculations in a specialized language, though: OpenCL's variant of C, or OpenGL's shading language. You can't just offload your Java code to a GPU, because GPUs don't understand Java bytecode. And there's no built-in equivalent to BigInteger
in either OpenCL or OpenGL.
Keep in mind that GPUs are designed for a specific kind of workload: running the same code simultaneously on many different data items. GPUs don't magically make math faster. Using OpenCL/OpenGL is only likely to help if your calculations can be split up into many parallel tasks, all running the same code at the same time.