This question is impossible to answer in any detail as you have provided no details about your code, not to even mention any code example.
But from my experience when people have problems with concurrency using Node and they are below 10,000 concurrent connections which you should handle easily (see this answer) then the usual suspect is that they are using blocking function calls.
Are you using any blocking code in your app? If so then here's your problem. If you have any blocking code in your Node app (like calling functions with "Sync" in their name) then you will have serious problems with concurrency. You should never use blocking functions anywhere else than on the first tick of the event loop (and if you don't know what it means then you should never use them at all).
Also, since you write that you're using Node for Java Web application it can be more complicated than that. If your Node app is connecting to Java app that spawns a new thread for every connection then the bottleneck may be in your Java app and not your Node app. As you provide no details about our architecture, it's impossible to say. But one this is for sure: if you want to handle massive concurrency then you cannot use threads. You need to use event loops. There's a reason why nginx or Redis are single-threaded. Threads don't scale, especially for I/O-bound use cases. Profile your code and see if it's your Java or Node code that needs fixing.