1

I am building a Jenkins job via the REST API and need to pass a huge JSON payload to the build - above 2MB

Currently, with less than 2MB JSON payload and using Multi-line input Parameters, I can successfully pass the JSON to the job.

My current usage looks like this

curl -sg -X POST "http://[xxxx:xxxx:xx:xxxx::a]:9090/job/Planner/buildWithParameters" --user admin:xxxxxxcbd73258xxxxxxxxxxx --data INPUT_LINE=$MY_JSON -H 'Accept: application/json'

But anything above 2MB, I get the following error

"error=7, Argument list too long
    at java.base"

which corresponds to getconf ARG_MAX -> 2,097,152 on my machine

I am currently looking for a workaround. it seems the ENV variables have a max value which Jenkins is using as build parameters

The solutions given here seems to be also using BuildParameters which wil end up as ENV variables, hence not suitable for setting huge payloads What is the format of the JSON for a Jenkins REST buildWithParameters to override the default parameters values

electricalbah
  • 2,227
  • 2
  • 22
  • 36
  • How does your JSON look like? Does it have whitespace, if yes, remove it. Also, how are you passing JSON to the API, does it go compressed or uncompressed? – kiner_shah Nov 24 '21 at 04:21
  • No white space, there are no escaping issues. The error I am seeing seems to be a reasonable error coming from exceeding the maximum value of the environment variable. I am just wondering if there are plugins or some other workaround out there – electricalbah Nov 24 '21 at 04:28
  • Why do you need to pass JSON as an argument? Can't use instead pass the JSON in the request body? Can you check this link: https://stackoverflow.com/q/18611903/4688321? – kiner_shah Nov 24 '21 at 04:31
  • @kiner_shah that would be really awesome but then how to recover the JSON object from within my Jenkins job? – electricalbah Nov 24 '21 at 04:35
  • Does this answer your question? [What is the format of the JSON for a Jenkins REST buildWithParameters to override the default parameters values](https://stackoverflow.com/questions/51508222/what-is-the-format-of-the-json-for-a-jenkins-rest-buildwithparameters-to-overrid). See the accepted answer, may help. – kiner_shah Nov 24 '21 at 04:37
  • @kiner_shah thanks for helping, however both solutions given by the accepted answer are using buildParameters as well – electricalbah Nov 24 '21 at 04:43
  • Accepted answer has two ways: one using /build and one using /buildWithParameters. With /build you can pass parameters as JSON in the request body. – kiner_shah Nov 24 '21 at 05:48
  • @kiner_shah as the answer states "various API calls to trigger parameterized builds". You must first define the parameters in the Jenkins GUI, then trigger /build with json. Hence jenkins will set the JSON parameter in the ENV variable and that will exceed the ENV Max limit – electricalbah Nov 24 '21 at 05:55
  • I don't understand your requirement? Can you post an example JSON which you need to send? – kiner_shah Nov 24 '21 at 05:58
  • this is the payload for 2 items in the json. as I mentioned early, it can work for less than 2MB around 800 items in the json. from 1500 items in the json it starts to fail as the payload size exceeds 2MB. [{"ipaddress": "192.168.45.1","hostname":"JVX001"},{"ipaddress": "192.168.45.1","hostname":"JVX002"}] – electricalbah Nov 24 '21 at 06:06
  • Why would you set the JSON as ENV variable? What is that ENV variable used for? – kiner_shah Nov 24 '21 at 11:20
  • @kiner_shah I am not setting anything as an environment variable. That is how Jenkins work with Build parameters. Jenkins will internally set your build parameters as ENV variables – electricalbah Nov 25 '21 at 04:33
  • Exactly, why do you need such a large JSON as a Jenkins build parameter? I am not sure if it is possible to increase the parameter size limit. Maybe these links help: [Link1](https://stackoverflow.com/q/50831470/4688321) [Link2](https://stackoverflow.com/q/59861259/4688321). But, for better scalability, I think the job structure should be re-designed so that you don't have to pass such large parameters. – kiner_shah Nov 25 '21 at 06:51

0 Answers0