Defining a variable in shell script portion of Jenkins Pipeline

Question

I'm trying to dynamically define a variable I use later in a some shell commands of my Jenkins pipeline and it's throwing an exception. I even tried to predefine the variable from an environment section to no avail. Is this a prohibited operation? My other variable myVar seems to work fine, but it's a constant through the pipeline.

pipeline {
    agent any

   environment {
     py2Ana=""
     myVar="ABCDE"
   }
    stages {
        stage('Stage1') {
            steps {
                sh """
                    echo myVar=$myVar
                    echo Find Anaconda2 Python installation...
                    py2Ana=`which -a python | grep --max-count=1 anaconda2`
                    if [[ -z "$py2Ana" ]]; then
                        echo ERROR: must have a valid Anaconda 2 distribution installed and on the PATH for this job.
                        exit 1 # terminate and indicate error
                    fi
                """
            }
        }
    }

Exception

groovy.lang.MissingPropertyException: No such property: py2Ana for class: groovy.lang.Binding
    at groovy.lang.Binding.getVariable(Binding.java:63)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:242)
    at org.kohsuke.groovy.sandbox.impl.Checker$6.call(Checker.java:288)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:292)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:268)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:268)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:29)
    at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
    at WorkflowScript.run(WorkflowScript:21)

Possible duplicate of [What's the difference of strings within single or double quotes in groovy?](https://stackoverflow.com/questions/6761498/whats-the-difference-of-strings-within-single-or-double-quotes-in-groovy) — mkobit, Jul 18 '18 at 19:37
https://stackoverflow.com/questions/39982414/access-a-groovy-variable-from-within-shell-step-in-jenkins-pipeline/66637408#66637408 — Shubham Jain, Mar 15 '21 at 11:56

score 34 · Accepted Answer · edited Jun 14 '21 at 16:44

34

As @jxramos stated, Jenkins is trying to resolve the variables in the script. It interprets any $string as a variable that needs substitution. The solution is to escape the $ of the in-script variables, as follows:

pipeline { 
  agent any 
  stages {
    stage('test stage'){
      steps {
        sh """#!/bin/bash
            myvar=somevalue
            echo "The value is \$myvar"
        """
      }
    }
  }
}

edited Jun 14 '21 at 16:44

jxramos

7,356
6
57
105

answered Nov 11 '19 at 23:35

JoeAC

852
1
8
13

Good addition, I think I came to realize this escaping in a closely related context in another problem I stumbled across when still learning the Jenkinsfile syntax stuff for pipelines https://stackoverflow.com/a/52484384/1330381. – jxramos Nov 11 '19 at 23:49

jxramos · Answer 2 · 2021-06-14T16:45:08.167

There appears to be a variable substitution precedence that Jenkins enforces in a preprocessing step if you will. In other words there's no delayed expansion as one would find in the Windows batch file behavior with setlocal ENABLEDELAYEDEXPANSION. This explains what's going on, and here's the test pipeline I used to determine this:

pipeline {
    agent any

   environment {
     py2Ana="DEFAULT"
   }
   stages {
       stage('Stage1') {
           steps {
                sh """
                    echo py2Ana=$py2Ana
                    py2Ana=Initialized
                    echo py2Ana Initialized=$py2Ana
                """
            }
        }
    }
}

This yields the following console output...

Started by user unknown or anonymous
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Stage1)
[Pipeline] sh
[TestPipeline] Running shell script
+ echo py2Ana=DEFAULT
py2Ana=DEFAULT
+ py2Ana=Initialized
+ echo py2Ana Initialized=DEFAULT
py2Ana Initialized=DEFAULT
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Another restriction that this poses is that you truly cannot use dynamic variables in the sh portion of the Jenkins declarative pipeline script since Jenkins will first attempt to resolve all variables before execution. Thus the following will always yield an error

sh """
   for filename in /tmp/**; do
      echo filename=$filename
   done
"""

The error being...

groovy.lang.MissingPropertyException: No such property: filename for class: groovy.lang.Binding

One would need to define a script dynamically (after figuring out a way to escape the $ to write to file), or already have it in the source, to be executed.

thanks for this, I have a question though, imagine you want to use this variable later in your post section? like somewhere in my script I want to have a variable and then use it later in the post when I'm using the ms teams connector in my post section. would this be possible? — Miguel Costa, Oct 28 '20 at 11:28

score 5 · Answer 3 · edited Jun 14 '21 at 16:46

The error itself seems really to be caused by the assignment of an empty string.

However: Do you really need that environment variable to be defined in the Jenkinsfile? To me it looks like you just want to set and read the variable from within the shell script. But the way it's coded the if [[ -z "$py2Ana" ]]; then would never pick up the value set by the shell script - it would always want to use a property from the Jenkinsfile - which didn't work.

You could use if [[ -z "${env.py2Ana}" ]]; then for the if condition which would fix that error but it still would not pick up the value set by the previous line but always read the empty string set in the Jenkinsfile.

To solve this you could either enclose the string in single quotes for the whole string like (maybe you even want to get rid of the myVar then)...:

pipeline {
    agent any

    stages {
        stage('Stage1') {
            steps {
                sh '''
                    echo Find Anaconda2 Python installation...
                    py2Ana=`which -a python | grep --max-count=1 anaconda2`
                    if [[ -z "$py2Ana" ]]; then
                        echo ERROR: must have a valid Anaconda 2 distribution installed and on the PATH for this job.
                        exit 1 # terminate and indicate error
                    fi
                '''
            }
        }
    }
}

... or add a backslash right before $py2Ana like:

pipeline {
    agent any

    stages {
        stage('Stage1') {
            steps {
                sh """
                    echo Find Anaconda2 Python installation...
                    py2Ana=`which -a python | grep --max-count=1 anaconda2`
                    if [[ -z "\$py2Ana" ]]; then
                        echo ERROR: must have a valid Anaconda 2 distribution installed and on the PATH for this job.
                        exit 1 # terminate and indicate error
                    fi
                """
            }
        }
    }
}

Either way without referencing env.py2Ana in the code I doubt the environment block in the Jenkinsfile still would make sense - that's why I removed it from the examples.

score -1 · Answer 4 · answered Jul 19 '18 at 03:38

-1

Just add a value to py2Ana

environment {
     py2Ana="1234"
     myVar="ABCDE"
   }

It doesn't create the variable in environment if you pass a empty string :)

answered Jul 19 '18 at 03:38

rohit thomas

2,302
11
23

1

You're right that empty strings do not succeed in bringing a variable into scope, good catch. Even still this approach doesn't present an opportunity for my use case however since Jenkins seems to preempt the scripts by an upfront variable substitution with what's known at the scripts interpreter stage. I'll detail my findings in an answer below. – jxramos Jul 19 '18 at 18:09

Defining a variable in shell script portion of Jenkins Pipeline

Exception

4 Answers4

Linked