Why am I getting an exception when attempting to kickoff a multibranch build job from a regular pipeline job?

Question

I am doing something similar to the accepted response from How to trigger Multibranch Pipeline Jenkins Job within regular pipeline job? . I have a regular pipeline job that is attempting to kick of a multibranch pipeline job, for a particular branch that has already been scanned and registered.

The code looks like:

 rc = build job: mb_job_name, 
       wait: true

where mb_job_name contains the name of the downstream job plus the branch name.

The strange thing is that the downstream job actually starts and runs fine. I see the expected:

Scheduling project: playground-sandbox » Chris » test-jenkins-mb » dv_main_117_chris2

output. The problem is, in my regular pipeline job that kicks off the downstream job, just after the build job call, and the expected Scheduling... output, it throws an exception like:

an exception which occurred:
    in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
    in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@28280b21
    in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
    in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@48fc924d
    in field com.cloudbees.groovy.cps.impl.CallEnv.caller
    in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@3b112c7a
    in field com.cloudbees.groovy.cps.Continuable.e
    in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable@a5e88d8
    in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
    in object org.jenkinsci.plugins.workflow.cps.CpsThread@6faa034a
    in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@d6a022e
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@d6a022e
Caused: java.io.NotSerializableException: groovy.json.internal.LazyMap
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:926)
    at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
    at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
    at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
    at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
    at java.util.HashMap.internalWriteEntries(HashMap.java:1793)
    at java.util.HashMap.writeObject(HashMap.java:1363)
    at sun.reflect.GeneratedMethodAccessor124.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
    at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
    at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
    at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
    at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
    at java.util.concurrent.ConcurrentSkipListMap.writeObject(ConcurrentSkipListMap.java:1437)
    at sun.reflect.GeneratedMethodAccessor146.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
    at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
    at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
    at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
    at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.lambda$writeObject$0(RiverWriter.java:144)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:237)
    at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:143)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:553)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:530)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:517)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:441)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:136)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

In a general sense, I understand that this means that I have code/variables that Jenkins can't serialize, but its unclear to me what it is.

I tried changing my code to the simpler form of:

build mb_job_name

and I also tried replacing the mb_job_name variable with a string literal with the name of the job, but no improvement.

update

It looks like it has something to do with the fact that I have the following code just before the build call:

def jsonSlurper = new JsonSlurper()
def pl = jsonSlurper.parseText(payload)
def repo_name = pl.repository.name

If I remove def jsonSlurper = new JsonSlurper() then I no longer get the exception. Unfortunately, I need to parse some JSON so I kind of need to create the object. Probably need to use the @NonCPS annotation somehow.

Use `readJSON` utility step instead of `JsonSlurper`. – MaratC Feb 20 '21 at 21:40 — MaratC, Feb 20 '21 at 21:40

score 2 · Accepted Answer · answered Feb 20 '21 at 09:13

If you want to use JsonSlurper inside the pipeline code, use JsonSlurperClassic instead. The difference is that the second one produces LinkedHashMap, a serializable object. The first one produces LazyMap which is not serializable.

There are different ways to parse JSON in the pipeline code, you can check this blog post for more information - https://e.printstacktrace.blog/how-to-catch-curl-response-in-jenkins-pipeline/#using-groovy-json-jsonslurperclassic-to-parse-the-response

Jon S · Answer 2 · 2021-02-23T06:50:01.103

1

As you mentioned, you need to use @NonCPS to tell Jenkins CPS that it is in a state where it can't serialize the program state. This is done by moving the non-serializable code into a function:

@NonCPS
def parseRepositoryName(payload) {
    def jsonSlurper = new JsonSlurper()
    def pl = jsonSlurper.parseText(payload)
    return pl.repository.name
}

Add this to the end of your code, and replace the old json parsing code with a function call to the new function:

def repo_name = parseRepositoryName(payload)

edited Feb 23 '21 at 06:50

answered Feb 20 '21 at 17:42

Jon S

15,846
4
44
45

Well we tried this also. but it didn't worked. maybe it totally depends on jenkins version also. – Dashrath Mundkar Feb 20 '21 at 18:08
If you still get serialization errors, then you have more code that needs to be inside NonCPS block. – Jon S Feb 20 '21 at 18:14
1

Thank you. This solution worked also, but I choose the use of JsonSlurperClassic as the best answer because it required less code changes and more flexibility. I did have to change the last line of the function to `return(pl.repository.name)` . The existing `return = ...` code resulted in an error. – Chris Ratcliffe Feb 22 '21 at 14:56
Fully understand thanks for pointing out the errer, I've update my answer. – Jon S Feb 23 '21 at 06:51

score 0 · Answer 3 · answered Feb 19 '21 at 21:33

0

The multibranch Job folder structure in Jenkins is different than a regular job and to trigger that from a remote job you have to specify 'JOBNAME/BRANCHNAME'. Might be you have mistaken on that format, Hence it's erroring. Take a look on my below example.

node {
    stage('call multibranch pipeline') {
        def res = build job:'test0/main'
    }
}

in my example, test0 - is the multibranch job name and main is the branch. I have not used wait: true because the wait default is True in build DSL

Output

answered Feb 19 '21 at 21:33

Samit Kumar Patel

1,932
1
13
20

Thanks. Yeah, I originally had that issue, and if you just specify the job name without the branch, it kicks off a scan, which is actually quite useful for me. But that doesn't cause the exception. I only get the exception when I specify it as "job/branch". Its interesting that you don't see the error. I am basically doing the same thing. I see the Scheduling project: line in my output, but not the "Starting building:" line. – Chris Ratcliffe Feb 19 '21 at 21:58

Why am I getting an exception when attempting to kickoff a multibranch build job from a regular pipeline job?

3 Answers3