I am running into issues when I try to run a python build script from a Jenkins Pipeline, which I don’t get when I run the same script on the same machine outside of Jenkins. I am wondering if my approach has some obvious flaws or if someone has some insights into the issue I am running into.
My Jenkins setup looks like this:
- I have a windows jenkins controller
- The controller uses the EC2 plugin to allocate and spin up build agents also running on windows
- The controller connects to the agent using Winrm, using the Administrator user to connect
- The Build agents are running on a custom windows AMI i have put together that contains everything needed to run UnrealEngine builds
- The java version on the agent:
java 17.0.6 2023-01-17 LTS
Java(TM) SE Runtime Environment (build 17.0.6+9-LTS-190)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.6+9-LTS-190, mixed mode, sharing)
- Intended Flow: When a build is triggered, the controller spins up an agent on a ec2 machine, which pulls down the project, runs the build and then uploads the completed builds to an S3 bucket
I have a Python build script that runs the various steps of my UE4 builds that I can start through powershell. If I run this script on the EC2 machine manually, the builds work just fine without issues. When I use the same powershell command in my Jenkins pipeline script to build the project, I run into issues that I never get outside of jenkins.
I expected the builds to work the same when called from jenkins, as when I try the build script manually on the agent by using RDP to log onto the machine and running the command from powershell. However, the builds from Jenkins fails with error that look like this:
c1xx: error C3859: Failed to create virtual memory for PCH
c1xx: note: the system returned code 1455: The paging file is too small for this operation to complete.
c1xx: note: please visit [https://aka.ms/pch-help](https://aka.ms/pch-help) for more details
c1xx: fatal error C1076: compiler limit: internal heap limit reached
The errors seem to change depending on the build run, but always seem to concern memory limits, which the instance shouldn’t have. I spin up instances with 64gb of physical memory and I’ve assigned a 64gb of virtual memory (paging file size) as well. Again, the thing that confuses me is the lack of this issue when running the build manually on the agent machine. The memory utilisation on the machine during the build step that fails is barely above 10%. Both during the jenkins step and when called manually.
I’ve looked into this for the last couple of days and I haven’t gotten closer to a solution. Based on what I’ve found, you can adjust the JVM memory limit using the -Xmx and -Xms options. I’ve set those to these values: -Xmx16G -Xms8G which should be plenty to run the builds but i still run into the same issue. I've had a look at the env variables available during the Jenkins builds and it looks like the agent should have access to all required parts.
I feel like I am missing something obvious. I am relatively new to using Jenkins so any insights into what could be happening here would be greatly appreciated! I would love to learn more about the differences of running a build on the machine vs what happens with a jenkins build. I am more than happy to provide any additional info that could be useful.
