-3

The exact question I wanted to ask has already answered here. Still I just want to explore few more possibilities (if there are any).

Scenario: My application is thread based data centric web-app and the amount of data gets decided at run time by User. A user can request some data operation, which triggers multiple threads, each thread transport own data. Sometimes the data selection crash the application with OutOfMemoryError i.e. insufficient space to allocate new object in the Java heap. When there are multiple users using the application concurrently and most of them request big data operations this situation (OutOfMemoryError) is more likely to occur.

Question: Is there a way that I can prevent whole application being crashed? I can limit the amount of data being pulled in memory but if there is a better way than this? Even after limiting the amount of data per user, multiple concurrent user can generate OutOfMemoryError. One user can be put on hold or exit rather than all survive.

Community
  • 1
  • 1
manurajhada
  • 5,284
  • 3
  • 24
  • 43
  • think about it this way: if you arrive to a point where you're out of memory, _something_ needs to either be cleared/overwritten or they simply need to wait until previous tasks are done in the first place. Ideally, you'd be able to get an estimation of the amount of memory for the task at hand. Try posting some more specifics to your question, as there are probably better solutions with the actual problem at hand. – Rogue May 08 '17 at 19:12
  • @Rogue Thanks but the points mentioned by you are already known to me. My concern is about concurrency i.e. if a User is causing problem to the application. Is there a way that I can prevent other users from him? – manurajhada May 08 '17 at 19:16
  • @Rogue I understand, your comment make sense. – manurajhada May 08 '17 at 19:20

1 Answers1

1

Consistently I have experienced the following point:

  • Stream large data out. Combined with a GZIPOutputStream on Accept-Inflate (maybe as servlet filter). This can be done for the file system and the database. Also there exist (URL based) XML pipelines where parts of the XML can be streamed.

This lowers memory costs, a stream may be throttled (articially slowed down.

  • PDF generation: optimize the PDF, repeated images only stored once, sensible font usage (ideally the PDF fonts, otherwise embedded fonts).

  • Office documents: the OpenOffice or Microsofts xlsx/docx variants.

In your case:

Have every process be combinable and stream there result to one output stream: a branching pipeline of tasks. If such a task might be called with the same parameters, yielding the same data, you could use parametrized URLs and cache the results.

I am aware this answer might not fit.

Joop Eggen
  • 107,315
  • 7
  • 83
  • 138
  • What if I start each new http request as new process? will it be efficient? By this way each user request will have no memory preemption from each other? – manurajhada May 12 '17 at 08:00
  • The normal way is to use a thread pool, reusing threads, Normally no problem with garbage collection, and quite an efficient restrictive mechanism. **Memory mapped files** can serve in some cases to evade using the java heap. Using a database of course scales good too. – Joop Eggen May 12 '17 at 20:33