0

Just a disclaimer, I am an IT person not a developer so I made need a bit more explicit explanations than you are used to... I apologize in advance.

I am being asked to fix an issue with one of our deployment jobs in Jenkins. The deploy job uses a chef cookbook and knife to .zip the compiled .wars and then copies the .zip to the test server. The job was previously working but the project files have grown much bigger and now when we run the job it fails to copy the .zip file when running the batch script.

It fails with the error

ERROR: NoMemoryError: failed to allocate memory

This is really strange because our Jenkins server has plenty of ram... I checked utilization while running the job and it doesn't go over 4-6GB total and we have 24gb assigned to the virtual machine that hosts Jenkins.

This seems to be some sort of artificial limitation because if I manually build the .zip and eliminate all the imported library .dlls and reduce the size of the zip to under 500mb it works fine. But I would rather not mess with my developer's project files to reduce the size (they all get pulled down from BitBucket when the job builds the source... I have no idea what kind of downstream effects it might have to just remove them).

There has to be a way to get this to copy .zips larger than 500mb... this is 2017, 500mb is a pitiful limitation.

Any ideas on how to troubleshoot further? Any specific log files I should be looking at on the Jenkins server? The console output is the only place I see anything, and all that tells me is the memory allocation error.

jospeechmorph
  • 1
  • 1
  • Still in need of assistance, the question has not yet been answered. This is not an issue with the jvm heap space, something else is running out of memory. – jospeechmorph Sep 12 '17 at 20:08

1 Answers1

1

Jenkins is a Java application, and the Java Virtual Machine might have its own memory limit. you can increase the max heap size up to 2GB with

java -Xmx2048m ...

java -X will display the list of all available options

on a redhat system, you can simply append -Xmx2048m to JENKINS_JAVA_OPTIONS in /etc/sysconfig/jenkins and then systemctl restart jenkins

Gilles Gouaillardet
  • 8,193
  • 11
  • 24
  • 30
  • Ahh I should have specified we are running Jenkins from a Windows 2012 R2 server... I am willing to give this a go. Where would be the appropriate place to make this change on a windows Jenkins install be? – jospeechmorph Sep 08 '17 at 01:28
  • does https://stackoverflow.com/questions/5936519/how-to-give-jenkins-more-heap-space-when-it%C2%B4s-started-as-a-service-under-windows answer your question ? – Gilles Gouaillardet Sep 08 '17 at 01:34
  • Sort of... I was able to increase it to 1024m using the instructions on that thread. However if I set it to 2048m the windows service fails to start with error 1067 "the process terminated expectantly". But either 1024 is not big enough (it should be, the zip is 965mb... but that could be 965mib which might be more than 1024mb... idk, math and stuff). Either that or this isn't the issue because I am still getting the failed to allocate memory error when running the deployment job with it set to 1024. Where might I find more granular log info? – jospeechmorph Sep 08 '17 at 02:03
  • I increased it up to 1700m which is the max it will let me set for some reason. But that should be enough... yet I am still receiving the failed to allocate memory error. Any thoughts? – jospeechmorph Sep 08 '17 at 19:36
  • I don't think it is Jenkins itself that is failing to allocate memory. I installed the monitoring plugin and reran the deploy job and the memory utilization under "Java memory used" doesn't go over 120Mb. I am thinking that some other module being called is what is having the memory issue... but the console output isn't helping me narrow down exactly what is failing to allocate memory. *bangs head against wall* – jospeechmorph Sep 08 '17 at 22:53
  • I still need help, please please please please someone help me... when I google search this now, this thread is the first thing to pop up! – jospeechmorph Sep 12 '17 at 23:55