Declarative Pipeline with dynamic matrix axis values

Question

Hi I am trying to get a Jenknis-Declarative-Pipeline-Job work.

The Use-Case should be pretty simple:
I want to build multiple Plugins with the same Jenkins-pipeline. To do so I wrote a "JenkinsLibrary" with an interface the Plugins can use for parameters. One of this parameters is the axis-values.

The Problem I have is pretty similar to this reddist post.

I want to set the "values" of the "axis" of the matrix-build from a variable. I am out of Ideas, is this even possible?

So here is my example:
a "JenkinsLibrary" with a file my_library.groovy

def call(Map i_options)
{
// later I will parse the options to set the values, but currently the step before already crashes ...

    def axis_1_values = "axis_1_value_A"
// already tried the following alternatives ...
//    def axis_1_values = 'axis_1_value_A'
//    def axis_1_values = '''axis_1_value_A'''
//    def axis_1_values = ["axis_1_value_A", "axis_2_value_A"]
    pipeline
    {
        agent any
        stages { stage("stage A") { matrix {
            axes {
                axis {
                    name "axis_1"
                    // values "axis_1_value_A" // <- of course this works ...
                    // but I want to read it from a variable
                    values "${axis_1_values}" 
                    // I already tried more variants
                    // values ${axis_1_values}
                    // values axis_1_values
                }
                axis {
                    name "axis_2"
                    values "axis_2_value_A", "axis_2_value_B"
                }
            }
            stages {
                stage("another stage") { steps {
                    echo "hello world from ${axis_1} && ${axis_2}"
                } }
            }
        } } }
    }
}

A Plugin with a file "Jenkinsfile":

@Library("JenkinsLibrary") _

def options = [
    axis_values: "a_axis_value"
]

my_library(options)

I am getting the following error:

[...] Expected string literal but got "${axis_1_values}"

complete log:

    19: Expected string literal but got "${axis_1_values}" @ line 19, column 28.
                       values "${axis_1_values}" 
                              ^

1 error

    at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
    at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
    at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
    at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
    at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
    at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
    at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:761)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:718)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
    at org.jenkinsci.plugins.workflow.cps.global.UserDefinedGlobalVariable.getValue(UserDefinedGlobalVariable.java:57)
    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
    at sun.reflect.GeneratedMethodAccessor729.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:160)
    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:162)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
    at WorkflowScript.run(WorkflowScript:21)
    at ___cps.transform___(Native Method)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
    at sun.reflect.GeneratedMethodAccessor500.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:39)
    at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
    at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:28)
    at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55)
    at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16)
    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

My System: Jenkins-Version: Jenkins ver. 2.190.1
Declarative-Plugin-Version 1.5.0
Declarative Agent API: 1.1.1
Declarative Extension Points API: 1.5.0
(many other plugins, I think this are the important ones)

UPDATE
I still do not have a solution. But when I define a variable like this, I understand that I try to insert a "GStringImpl" to that. But I am unable to cast it as java-string.

def _str_1 = "a string"
def _str_2 = "${_str_1}"
println _str_1.class             // -> class java.lang.String
println _str_2.class             // -> class org.codehaus.groovy.runtime.GStringImpl
def _str_3 = "${_str_1}" as java.lang.String  
println _str_3.class             // -> class java.lang.String

But I still don't know how I can add a single value, because if I do

values "${axis_1_values}" as as java.lang.String     // error-msg: Expected string literal but got ${as}  
values ("${axis_1_values}" as java.lang.String)      // error-msg: Expected string literal but got ${"${axis_1_values}"  
values axis_1_values.toString()                      // error-msg: Method calls on objects not allowed outside "script" blocks.
values "${axis_1_values}".toString()                 // error-msg: Expected a symbol @ line 26, column 28.

This casting stuff may work for a single value, but I want to be able to add a list of values, like its possible in the default example which works ...
values "axis_2_value_A", "axis_2_value_B"

I am wondering, that this works in the "docker-agent"-part of the declarative pipeline.

agent { 
    docker {
        image "${_image}:latest"
        label "${_label}"
    }
}

But I in the code, I realized that the parsing of the docker-label is implemented separately ...

I have still no Ideas how to reach that goal within the matrix / axis features of the declarative pipeline.

(funfact: one of the main reasons why even doing this that the visualisation in the blue-ocean view for scripted-pipelines is still buggy and the issue seems to be ignored ...
https://issues.jenkins-ci.org/browse/JENKINS-53751 )

What are you trying to do in your `axes` block? Are you trying to initialize a `Map`? — Matthew Schuchard, Jan 21 '20 at 16:23
Hi @MattSchuchard I am not sure what you mean with `Map`. The axes are the build-variants (so the axes are "Plattforms", "compiler" and "architecture".) And the user can define for each plugin, which Plattform / Compiler / Arch it can be build by Jenkins. — tharilya, Jan 21 '20 at 16:30
So what is the code in the `axes` block supposed to be doing from a code perspective, and not a UI perspective? — Matthew Schuchard, Jan 21 '20 at 16:40
The code should generate all variants the sources of the plugin will be build. simplified example (just compiler and architecture): the user wants to build the following variants: "vc15-x64"; "vc16-x64"; "vc15-ARM"; "vc16-ARM" This should be achieved with 2 axis: axis_1_values "vc-15", "vc-16" axis_2_values "x64", "ARM" the `axes` block should create all possible variants of it (like written above) in the later pipeline I will download sources, execute a compile step and so on. This all works if the `values` in the `axis` entry are written plain. — tharilya, Jan 21 '20 at 16:51
Ok got it. Based on the error message, code, and documentation, I believe your problem is that you are attempting to use the syntax to interpolate a variable, but Jenkins Pipeline demands a literal string. You will either need to provide a literal string as the argument, or use the full `Matrix` class for dynamic values like you are trying to do. — Matthew Schuchard, Jan 21 '20 at 17:23
Ok, and how can I do that? How can I return a literal String from a Variable to the "axis" Parameter? — tharilya, Jan 21 '20 at 17:47

score 9 · Accepted Answer · answered Apr 28 '20 at 17:35

Dynamic axis values are not supported in Declarative Pipeline as of v1.6.0 of the plugin. The error message says it expects a "literal string" value. Perhaps the message should also say, "Template strings, arrays, variables, or function calls are not allowed."

This is by design, intended to keep pipeline authors from getting bitten by some internal assumptions of the pipeline engine. It might be possible to change but will require careful planning and testing.

Please file a JIRA at https://issues.jenkins.io/ and/or join the Jenkins Pipeline Authoring SIG and help implement this feature.

Thank you for your answer and the information. I created a ticket as you suggested.( https://issues.jenkins-ci.org/browse/JENKINS-62127 ). — tharilya, Apr 30 '20 at 16:07

Declarative Pipeline with dynamic matrix axis values

1 Answers1

Linked