0

This is an issue with I am trying to fix, maybe a bug in JRE please vote to reopen this question. It's very selfish approach to mark something off topic if you are not competent enough to understand some problem.

Java Version

java version "1.7.0_45" Java(TM) SE Runtime Environment (build 1.7.0_45-b18) Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)

Following both version of codes produces the same crash.

Client machine call a Servlet deployed in Tomcat 7 using a Server Maching that is running on Windows Server 2003 64-bit, Using Oracle 10g database and Hibernate 3.0.

When I try to put load on Servlet for load testing I wrote following two methods that just creates a thread and call Synchronous method but without any stacktrace of log My Process in Eclipse termiates.

I don't know the reason of crash if I create 500 threads then it crashes at 487 and If I create 1000 threads then it crashes at 874. I don't know what the real reason is. I have configured a StuckThreadDetectionValve to detect long running Threads.

https://stackoverflow.com/a/7484800/185022

I'll share all the updates here. Please give me a hint at least if you know that what could possibly be wrong.

** Both codes produces the same crash **

Code 1:

public class TokenTest {
    static int count = 0;

    public static void main(String[] args) {
        try {
            System.out.println("Start " + new Date());
            final ApplicationServiceClient app = WSClient.getClient("http://192.168.1.15:81/az-server", ApplicationServiceClient.class);
            final int max = 50;
            ExecutorService executor = Executors.newFixedThreadPool(max);

            for (int i = 1; i <= max; i++) {

                executor.execute(new Runnable() {
                    public void run() {
                        try {
                            System.out.print(++count +",");
                            app.verifyToken("7056451004350030976"); //Network Synchronous Operation, calls a Servlet that is deployed in another Server Machine, Using Tomcat 7.0
                        } catch (Exception ex) {
                            ex.printStackTrace();
                        }

                    }
                });
            }
            executor.shutdown();

            while (!executor.isShutdown()) {
                System.out.println("END " + new Date());
            }
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }   

}

Code 2:

public class TokenTest {
    static int count = 0;

    public static void main(String[] args) {
        try {
            System.out.println(new Date());
            final ApplicationServiceClient app = WSClient.getClient("http://192.168.1.15:81/az-server", ApplicationServiceClient.class);
            final int max=250;
            for (int i = 1; i <= max; i++) {
                Thread t = new Thread() {
                    public void run() {
                        app.verifyToken("7056451004350030976");//Network Synchronous Operation
                        System.out.print(count++);
                        System.out.print(", ");
                        if (count >= max)
                            System.out.println("\n" + new Date());
                    }
                };

                t.start();
            }

        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }



}

Note: The server is hosted on another Windows Server 2003 64-bit machine.

Updated It just terminates my Java process in Eclipse and If I give value like 5000 then it stuck on 2800 something and nothing happens neither it terminates my process. Here is a console snapshot.

Console

Update* Thank to S.Yavari. He led to the right direction. After configuring Stuck_Thread_Detection_Valve on Tomcat and interrupting the Threads that Stuck Threads I am able to generate load of 500K clients on server without any problem and Also giving a Sleep of 100 miliseconds before starting new request.

Below is the code :

public class StageOne {

    final ApplicationServiceClient appClient = WSClient.getClient(Main.url, ApplicationServiceClient.class);
    final SecureWebServiceClient secureClient = WSClient.getClient(Main.url, SecureWebServiceClient.class);
    final static List<Long> times = new Vector<Long>();
    String token = "";

    public String authenticateClient() {

        token = appClient.authenticateClient("az_y", "az", "0");
        // System.out.println("Registration Response: " + token);
        return token;
    }

    private void getDefaultDepartmentName() {

        String deptname = appClient.getDefaultDepartmentName();
        // System.out.println("Department: " + deptname);
    }

    private void getLocation() {

        List<String> location = appClient.listLocation();
        // System.out.println("Location: " + location);
    }

    private void registerClient_() {
        secureClient.setToken(token);
        String respon = secureClient.registerClient("Buddy AZ", "443", "njuy", "1");

        // System.out.println(respon);

    }

    static int count = 0;

    public static void main(String[] args) {

        int pool = 500000;

        final ExecutorService execotors = Executors.newFixedThreadPool(pool);

        for (int i = 0; i < pool; i++) {
            try {
                Thread.sleep(100);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            execotors.execute(new Runnable() {

                @Override
                public void run() {

                    long startTime = System.nanoTime();

                    StageOne one = new StageOne();

                    one.authenticateClient();
                    one.getDefaultDepartmentName();
                    one.getLocation();
                    one.registerClient_();

                    long EndTime = System.nanoTime();

                    // System.out.println("Time in Seconds ("+ (++count) +")" + TimeUnit.NANOSECONDS.toSeconds(EndTime - startTime));
                    times.add(TimeUnit.NANOSECONDS.toSeconds(EndTime - startTime));
                }
            });
        }
        execotors.shutdown();

        while (!execotors.isShutdown()) {
            try {
                Thread.sleep(10);
            } catch (Exception e) {
                // TODO: handle exception
            }
        }

        Collections.sort(times);
        long sum = 0;
        for (int i = 0; i < times.size(); i++) {
            sum += times.get(i);
        }
        long result = sum / Math.max(1, times.size());


        System.out.println("Min Response Time: " + times.get(0));
        System.out.println("Average Response Time: " + result);
        System.out.println("Max Response Time: " + times.get(times.size() - 1));

    }

}
Community
  • 1
  • 1
AZ_
  • 21,688
  • 25
  • 143
  • 191
  • 1
    Please post details of your JVM crash, including any error messages printed. – Robin Green Dec 22 '13 at 10:41
  • Nothing is printed, It just terminates my process in Eclipse. – AZ_ Dec 22 '13 at 10:49
  • 1
    Can it be reproduced outside of Eclipse? – Andrey Chaschev Dec 22 '13 at 10:52
  • 1
    So nothing is printed at all? Not even the `start` println in your code? – Robin Green Dec 22 '13 at 10:52
  • @AndreyChaschev No I haven't tried – AZ_ Dec 22 '13 at 10:54
  • 1
    What do you mean "give value like 5000"? what value are you talking about? – Robin Green Dec 22 '13 at 11:00
  • @RobinGreen Number of threads (Value corresponds to threads, hit and trial trying to find out the reason) – AZ_ Dec 22 '13 at 11:02
  • This is extremely confusing. What are code 1 and code 2? Which one is the one that crashes the JVM? – Robin Green Dec 22 '13 at 11:05
  • @RobinGreen I tried with both ways but the result is same. both codes produces same out put crash – AZ_ Dec 22 '13 at 11:06
  • When the JVM crashes you get a message printed to stdout (or err? well something) and find a log file in the working directory. But considering the serious race conditions in this example I doubt the JVM is to blame for problems – Voo Dec 22 '13 at 18:23
  • @Voo then what do you suggest can be root cause of problem. – AZ_ Dec 23 '13 at 04:37
  • I have given complete SSCCE. If any one having problem in understanding please see Code and Code 1 – AZ_ Dec 23 '13 at 04:38
  • @Az_ it's simple: if the process doesn't terminate the problem is in the verifyToken call, if the process fails with an error message and creates a core dump and log it's a JVM error, if neither of those happens it's just the fact that count++ in a multithreaded program will lead to lost updates. On a related note: always assume it's your own fault and not the compiler/framework/moon you'll be correct almost all the time and you'll be taken more seriously that way. – Voo Dec 24 '13 at 13:12

2 Answers2

3

I think this is just a Threading problem. Use this code:

public class Test2 {
    public static final Object LOCK = new Object();
    static int count = 0;

    public static void main(String[] args) {
        try {
            System.out.println(new Date());
            final ApplicationServiceClient app = WSClient.getClient("http://192.168.1.15:81/az-server", ApplicationServiceClient.class);
            final int max=250;
            for (int i = 1; i <= max; i++) {
                Thread t = new Thread() {
                    public void run() {
                        synchronized (LOCK) {
                            app.verifyToken("7056451004350030976");
                        }
                        System.out.print(count++);
                        System.out.print(", ");
                        if (count >= max)
                            System.out.println("\n" + new Date());
                    }
                };

                t.start();
            }
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

This method (verifyToken) must be thread safe, otherwise you will have so many problems using this method inside threads.

S.Yavari
  • 876
  • 8
  • 25
  • It will not help. I want to do load testing. – AZ_ Dec 23 '13 at 04:37
  • 1
    This is not a JVM problem. Crashing is inside `verifyToken` method. This can be a threading problem or maybe `verifyToken` can not handle 250 concurrent connections. Another solution for you: set timeout for your `verifyToken` method if you can. – S.Yavari Dec 24 '13 at 05:48
  • 1
    If your server has no response and your connection timeout is infinity, your code will wait for response without any activity. – S.Yavari Dec 24 '13 at 05:54
  • S.Yavari 'app.verifyToken("7056451004350030976");" calls Servlet deployed in another Server machine. – AZ_ Dec 24 '13 at 08:23
  • 1
    I know it calls a Servlet deployed in another server. And exactly because of this, I said that you shuld set timeout for this method connection. Crashing means that, Servlet on another server can not handle this count of concurrent connections or you have threading problems. To find out that which one of this problems is happens you have to set timeout for your connection. If you get a ConnectionTimeout, then problem is from Servlet, otherwise, problem is a threading problem from your code. – S.Yavari Dec 24 '13 at 13:09
1

In code 1, the code is not crashing. It is working as designed. There doesn't seem to be any problem with that code.

Robin Green
  • 32,079
  • 16
  • 104
  • 187
  • Because you don't have server code. It's really a hug system and a customer product so dear I fear that I can not leak any server side code. – AZ_ Dec 24 '13 at 08:38
  • OK, that's what you claim, but you haven't proven it. – Robin Green Dec 24 '13 at 08:41
  • I still don't understand what is going on, unfortunately. I'm afraid I'm out of patience and I won't be able to help you. – Robin Green Dec 24 '13 at 13:16