0

There are some ways to replay java 8 streams e.g. mentioned here https://dzone.com/articles/how-to-replay-java-streams. Now what if i want to use log huge java stream (200-300mb) and pass it forward for replay. That should work right? But what if i dont want my JVM occupy that much RAM at any given point of time. Is that still possible and whats is the solution for that?

AlexL
  • 33
  • 6
  • maybe should clarify that this is about mandatory logging of huge files for security purposes – AlexL May 07 '20 at 11:32
  • 2
    The very article you linked to says you have two options: store in an intermediate collection (consuming RAM), or regenerate it from source. So regenerate from the source. – Michael May 07 '20 at 11:52
  • i am afraid i dont have a source to regenerate from, i just have a stream as a starting point for me. – AlexL May 07 '20 at 12:12
  • Then it's already in-memory. – Michael May 07 '20 at 12:26
  • true, but we dont want to double that memory usage for additional read and i still wonder whats the best way to replay that. – AlexL May 07 '20 at 12:43
  • If it's already in memory, then repeat whatever it was that got you the first stream to get a second – Michael May 07 '20 at 12:45
  • but i cant repeat whatever that got me there, we risk to get a different result and thats unacceptable risk. – AlexL May 07 '20 at 12:49
  • So you need to store the intermediate result somehow. Mapdb if you care about heap space, maybe – Michael May 07 '20 at 12:55
  • Thanks, it looks very interesting. But in your opinion there is nothing that can be done within java streaming in these case? – AlexL May 07 '20 at 13:14
  • Well, you can store in an intermediate collection on the heap. I'd say that counts, but you have reasons for not wanting to do that. – Michael May 07 '20 at 13:17
  • No, that causes memory issues that is one of the driving forces for this. Strange thing i can't find any info about repeatable stream solutions outside of several integration platforms like Apache camel and Mulesoft 4 that have it built-in. – AlexL May 07 '20 at 13:31
  • 1
    A Stream does not occupy significant memory. A Stream just describes a processing pipeline. What is occupying memory, is some sort of data in a particular storage. Since a processing pipeline usually contains conversions, like map operations, there are alternatives regarding what to remember in which form, which can lead to entirely different memory consumption. That’s why it makes no sense to attribute the resulting memory consumption to the Stream or to ask a question about how to reduce the memory consumption, without giving any context about the actual data and operations. – Holger May 07 '20 at 13:39
  • Thank you Holger! i understand, but the question was more about ways to replay java stream than about memory consumtion. – AlexL May 13 '20 at 05:41

0 Answers0