How to handle load in a high traffic Java web application?

Question

I am developing a high-traffic java web application. Currently I'm defining its architecture. I have some questions:

• What are the key elements that we should consider while designing the architecture to handle load?
• How to prevent web application from getting down or how to ensure high availability?

I expect hundreds of user logging in at a time. I am planning to keep data in application scope (static variables) to avoid lookups in the database. In session I am planning to store 5kB of data.

Answer 1

I guess key elements I would consider would be:

• Do you need cluster aware app servers?
• Do you need hardware or software based load balancers?
• What parameters will you record to determine the health of a server?
• Will you load balance your DBs?
• Will your app be I/O heavy? CPU heavy? Both?
• Are you looking to utilize a lot of web traffic? If you're averaging 5K of data per user, how many users will you be able to handle before you consider a link saturated?
• Scale up vs scale out. Determine which one your app does best and exploit it.
• Will there be a lot of DB transactions?
• Will you be using Shared Storage?

I hope this helps.

Answer 2

Some of my considerations and best practices :

• Use SOLID principles, not all are applicable for all projects but especially single responsibility and open/closed principle to helps you to expand/customise software easier against changing needs and growing traffic
• Keep rarely changing configuration data in memory database, if changes are required update relational database first and call Restfull API to refresh memory objects
• Define and partition logic for your big data by estimating the product growth to run your search queries faster. e.g. If mobile number is the key value applying modulo operation to decide the partition id by considering the relational or in memory database single db object size constraints
• Keep database updates on the same row minimum by serializing the update operations. If data state is being updated in short intervals, keep it in memory and using scheduled tasks to do batch updates to relational database.
• Use full power of available logging libraries to log what is important and meaningful, prefer asynchronous logging.
• Do history recording via separate queue, use batch inserts and consume it via scheduled jobs
• Consider using asynchronous API logic for faster execution, expect the request with session Id or return tracking id to request and send responds when the execution done by parallel jobs.
• Implement retry logic for failed notifications to consumer endpoints
• Define each software subsystem independent and self-sufficient. Subsystems should communicate and share data through API not through shared memory or disk storage.

Answer 3

Scalability is very important to a system expected to handle growing traffics. And stateless is very important to implement an easy scale system. Checkout http://www.playframework.org/, a stateless web framework. Using playframework with mongodb (via PlayMorphia module) you can easily implement a scalable web application.