1

We are using Tomcat7 with Java 7. We are getting a throughput of around 500 req/sec. But with multiple instances in place and only 20 request is served by single instance.

The problem is as it goes over 20 requests on single instance response time adds up.

As for an example : -

***1. if 1st request takes : - 5ms (Millisecs)

  1. then 2nd request takes: - 10ms

  2. 3rd : - 15ms and so on.......***

I tried making it constant but it is not working.

What have we tried : -
1. Changing connector from BIO to NIO in Tomcat7.
2. Increasing maxThreads = 1000.
3. Increasing Cores from 2 cores to 10 cores.
4. Async API implementation.

None of the above worked.

What We want to achieve :-

  1. We would like to have constant response time of API.
  2. 100 req/sec in one single instance.

-----------------<>---------------<>--------------------------<>----<>--------------

Edit:-

Ok After a day of reading the code and trying different things. Found the main reason because of which performance is Low. This is due to InputStream read() function blocking nature. When java reads the o/p from socket . Socket returns the o/p in inputstream. Inputstream.read() is a blocking method in java (Link attached below). Still have no idea how to fix it.

Link : - http://javarevisited.blogspot.in/2012/02/what-is-blocking-methods-in-java-and.html#axzz4kWX7K7Yc

Have tried many methods of Java NIO but nothing is working so far.

Thanks in advance.

Diptendu
  • 21
  • 1
  • 9

1 Answers1

1

Look at the memory consumption, networking, and maybe start a profile. On what kind of hardware are you running? Make sure your java memory parameters are being passed correctly to tomcat. Enable garbage collection logs.

https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/memman.html

CATALINA_OPTS vs JAVA_OPTS - What is the difference?

Also, please describe the flow, there might be nothing bad with your tomcat, for example, if you are using a database that limits to X connections, then you are being trottled by the database.

What are the most common requests? What those request are doing?

galusben
  • 5,948
  • 6
  • 33
  • 52
  • 1
    Hi gba, Thanks for your reply. We are not using any Database(DB) in the process. Flow of the program is : - nginx accepts the Request (Port 80)---> Tomcat 7(Port 8080) (NIO Connector) ---> Unix Socket ((Running on Localhost) Takes up around 20ms to 30 ms to respond for 1 requests) -- > return back the result. **We have noticed that reading the request from unix socket i.e. Inputstream(Is the main culprit in increasing the response time. But how to avoid using inputstream(I dont know). If we have to use input stream then how to make it avoid taking so much time.)** . Thanks,Diptendu – Diptendu Jun 19 '17 at 07:59
  • Hi gba, other stats : - memory consumption is almost 30 % (Max) for 20 connections going to one machine. Hardware: - Single instance 2 cores with 8 GB Ram. We are passing Java params through setenv.sh file under Tomcat7. Request made to server takes a row of data process it using unix Socket (Inside socket we have Python program running at all times) and returns back to client with a row of json data. – Diptendu Jun 19 '17 at 08:07
  • 1
    I am still not sure what is the flow, please elaborate exactly what happens from the time tomcat receives the call and until it return a response. Also, check your configuration for connection limits in both nginx and tomcat. – galusben Jun 19 '17 at 08:22
  • Hi gba, nginx is set to connect to 2000 connections and tomcat maxthreads = 1000 . – Diptendu Jun 19 '17 at 11:30
  • look at the maxConnections parameter in https://tomcat.apache.org/tomcat-7.0-doc/config/http.html it is different than the max threads. Also, I still need a more elaborated explanation of the flow to help you out. I do not understand what the API is doing. Do you have anything synchronized? – galusben Jun 19 '17 at 11:37
  • Hi Gba, maxConnections is 20000 . I did try maxConnections as -1 at the beggining but dint't work. I have a method to call unix-socket i think that is synchronized . – Diptendu Jun 19 '17 at 12:32
  • Hi gba, Flow of the API : - Tomcat accepts the request ---> Request body contains a row of JSON Data. --> That is read by Java as inputstream from request httpservlet . --> Row of data is passed to Unix-Socket --> Unix-socket returns the result in inputstream. That is read by Java and converted to Proper JSON. Using JSON Parser. ---> Then JSON Data is sent as response to client in the same request. Hope this flow clears things. – Diptendu Jun 19 '17 at 12:41
  • Seems like you are locking the write operation to the socket. That causes your problems. Also, if I understand correctly there is a python script processing the socket output, make sure it uses multiple threads. – galusben Jun 19 '17 at 13:51