1

Running tests parallel gives java heap out of memory. But when I run them not parallel, there is no memory issues. This is the error- [java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:3332) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448) at java.lang.StringBuilder.append(StringBuilder.java:136)

Kumar112
  • 224
  • 1
  • 5
  • 14

4 Answers4

2

Whenever an object is created the object is allocated memory from Heap, when the object is not needed any more the memory goes back to Heap space. If tests are run in parallel there will be different threads adding objects to the heap simultaneously, and you run out of heap space. When you run them single threaded, once a test finishes its objects are removed from the heap, therefore there's space in the heap for the objects in the following test.

To fix it try to increase the heap space used to run the tests.

rvelaz
  • 553
  • 7
  • 14
  • I already reduced my threads from 5 to3 to now one. Part of the puzzle is - same set of tests were passing with 5 threads not long ago. And the heap size right now is around 7 gig which is not less. – Kumar112 Jan 25 '18 at 18:33
  • @Kumar112 the answer is simple - you have more tests now :) Also, are you using the maven-cucumber-reporting plugin ? there is a known issue when you are using that to generate reports. Karate's memory consumption is minimal unless you have large payloads or are doing file-upload. – Peter Thomas Jan 26 '18 at 11:11
  • @Peter Thomas Tests didn't increase dramatically. But, I do have some tests with File uploads. I will try removing them and see if it solves the issue. – Kumar112 Jan 26 '18 at 22:36
1

Yes, try to increase the heap space: https://stackoverflow.com/a/16969122/143475

Also you could reduce the number of parallel threads. It sounds like you are trying to use too many.

EDIT: this problem was fixed in v0.8.0

EDIT2: Karate 1.0 should be greatly improved

Peter Thomas
  • 54,465
  • 21
  • 84
  • 248
  • I removed all tests related to file Uploads. I made the the parallel runner to only run on single thread. Even then, i see the heap size issue. – Kumar112 Jan 29 '18 at 20:45
  • please upgrade to 0.7.0.RC6 and re-try: https://github.com/intuit/karate/wiki/Upgrading-To-0.7.0 – Peter Thomas Jan 30 '18 at 02:45
  • Didn't work with 0.7.0.RC6. How ever it is WORKING with v0.5.0. Same test cases and same runner (parallel runner with 1 thread) works with v0.5.0. Please let me know what all info you need to see whats the problem. – Kumar112 Jan 30 '18 at 21:46
  • correction. with v0.5.0 , the same set of test cases with 5 parallel threads works just fine. Anything over 0.5.0 version, doesn't work. – Kumar112 Jan 30 '18 at 21:59
  • 1
    @Kumar112 ok, I have no more ideas. so unless you can narrow down the problem so that I can replicate this in my environment, I CANNOT do anything. use your creativity and try to comment out / tag remove tests one by one etc. all the best. karate is working fine in many locations around the world. – Peter Thomas Jan 31 '18 at 00:27
  • also set file.encoding and see if that makes a difference: https://github.com/intuit/karate#fileencoding - also PLEASE upgrade the Java version: https://github.com/intuit/karate/wiki/Upgrading-To-0.7.0 – Peter Thomas Jan 31 '18 at 00:29
  • I do have the file encoding set already and java Version is higher than the requirement you specified. I will try to narrow down the scenario and let you know. BTW Karate is working wonders for me too and absolutely love it.Thanks for the great product. – Kumar112 Jan 31 '18 at 14:55
  • @Kumar112 thanks ! I will help any way I can. BTW where are you based in the US ? – Peter Thomas Jan 31 '18 at 15:02
  • @Kumar112 you can e-mail me here: https://twitter.com/KarateDSL/status/937271659785961477 – Peter Thomas Jan 31 '18 at 15:03
  • 1
    I am in Virgina,US. Will be sure to email you i need to. Thanks for help! – Kumar112 Feb 02 '18 at 14:51
  • I do have few tests with large payloads. If that's the reason for the heap size issues, what could be they be alright with v 0.5.0? Any thoughts? Thanks!! – Kumar112 Feb 05 '18 at 16:03
  • yes I have a few thoughts, but hard to put down here. can you please refer to these 2 issues: https://github.com/intuit/karate/issues/264 and https://github.com/intuit/karate/issues/246 - especially making log level INFO and try to configure `printEnabled` as false – Peter Thomas Feb 05 '18 at 16:30
  • I changed the logging from DEBUG to INFO and that did the trick. Thanks for your help!! – Kumar112 Feb 05 '18 at 20:58
  • @Kumar112 an update. this was fixed in 0.8.0 so you can still have DEBUG and not run into this issue any more :) – Peter Thomas Jul 21 '18 at 05:24
  • 1
    I did see the release notes of 0.8.0 as i follow you on twitter and linkedin (i go by anilkumar.moka) @Peter Thomas. i will test it out and let you know. Thanks. – Kumar112 Jul 23 '18 at 21:03
  • 1
    I tried enbling logging but tests take way too longer to complete. With out logs, test would completed in about 3 mins vs 14 mins with logs. I disabled them again. Let me know if you need any specific things from me to help fix the issue. – Kumar112 Jul 24 '18 at 18:23
  • 1
    @Kumar112 okay, at least no more OOM. but wow I would not have expected that much - it probably means your HTTP payloads are really verbose. nothing for now, for the next version I'll try add a feature to provide more data on parallel runs as to which feature is taking time – Peter Thomas Jul 24 '18 at 19:00
1

I had to turn off my logs (karate 0.9.6):

karate.configure('report', { showLog: false});

This is already fixed in karate 1.0.0

Corleone
  • 43
  • 5
-1

This is happening because you are doing parallel execution and karate gives huge logs, make the below changes and make parallel thread 1,2 or max 3.

  <root level="info">
   <!--<appender-ref ref="STDOUT" />-->
    <appender-ref ref="FILE" />
  </root>
Sudheer Singh
  • 555
  • 5
  • 10
  • karate 1.0 should not see these issues any more. instead of these hacks I recommend that you first test 1.0 and if you still see the problem, submit a way to replicate: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue – Peter Thomas Apr 02 '21 at 13:19