Unable to use a variable within a "sh clause" in a Jenkins pipeline, what could be the reason?

Question

I'm writing a Jenkins pipeline.

I'd like to set the GRADLE_USER_HOME variable with the current directory/.gradle and so I'm populating the cwd variable with the pwd() function which is supposed to return the current working directory.

stage 'Host preparation'
    cwd = pwd()
    withCredentials([ // Use Jenkins credentials ID of artifactory
      [$class: 'UsernamePasswordMultiBinding', credentialsId: artifactory_creds, usernameVariable: 'A_USER', passwordVariable: 'A_PASS'],
    ]){
    sh """
        export NDK_VER="r12b"
        export SDK_VER="r24.4.1"
        export GRADLE_USER_HOME='${cwd}/.gradle'
        echo "CWD OK"
        export NDK_DIR='$GRADLE_USER_HOME/android-ndk-$NDK_VER'
        export SDK_DIR='$GRADLE_USER_HOME/android-sdk-linux'
        export PATH='$PATH:$GRADLE_USER_HOME:$GRADLE_USER_HOME/android-ndk-$NDK_VER:$GRADLE_USER_HOME/android-sdk-linux'
        if [ ! -d $GRADLE_USER_HOME ]; then
            mkdir $GRADLE_USER_HOME
        fi

        if [ ! -f "$GRADLE_USER_HOME/android-sdk-${SDK_VER}-linux.tgz" ]; then
            curl -o "$GRADLE_USER_HOME/android-sdk-${SDK_VER}-linux.tgz" https://dl.google.com/android/android-sdk_$SDK_VER-linux.tgz
        fi

        if [ ! -f "$GRADLE_USER_HOME/android-ndk-${NDK_VER}-linux-x86_64.zip" ]; then # Checks if the sdk tarball exists on system
            curl -o "$GRADLE_USER_HOME/android-ndk-${NDK_VER}-linux-x86_64.zip" https://dl.google.com/android/repository/android-ndk-$NDK_VER-linux-x86_64.zip
        fi

        if [ ! -d "$GRADLE_USER_HOME/android-ndk-$NDK_VER" ]; then
            cd "$GRADLE_USER_HOME" && unzip -o "android-ndk-$NDK_VER-linux-x86_64.zip"
        fi
        if [ ! -d "$GRADLE_USER_HOME/android-sdk-linux" ]; then
            tar -xzf "$GRADLE_USER_HOME/android-sdk-${SDK_VER}-linux.tgz" -C $GRADLE_USER_HOME
            mkdir "$GRADLE_USER_HOME/android-sdk-linux/extras"
        fi

        echo "Installing updates"
        export SDK_TOOLS="${GRADLE_USER_HOME}/android-sdk-linux/tools"
        # Downloads the required SDK tools
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 2     # Android SDK Tools, revision 25.2.2 rc1
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 3     # Android SDK Platform-tools, revision 24.0.2
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 4     # Android SDK Build-tools, revision 24.0.2
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 6     # Android SDK Build-tools, revision 24
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 7     # Android SDK Build-tools, revision 23.0.3
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 8     # Android SDK Build-tools, revision 23.0.2
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 10    # Android SDK Build-tools, revision 23 (Obsolete)
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 30    # SDK Platform Android 7.0, API 24, revision 2
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 31    # SDK Platform Android 6.0, API 23, revision 3
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 53    # Android TV Intel x86 Atom System Image, Android API 24, revision 6
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 54    # Android Wear ARM EABI v7a System Image, Android API 24, revision 1
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 55    # Android Wear Intel x86 Atom System Image, Android API 24, revision 1
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 57    # ARM EABI v7a System Image, Android API 24, revision 6
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 58    # Intel x86 Atom_64 System Image, Android API 24, revision 6
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 59    # Intel x86 Atom System Image, Android API 24, revision 6
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 153   # Android Support Repository, revision 36
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 160   # Google Play services, revision 32
        # echo "y" | "${SDK_TOOLS}/android" update sdk -u -a -t 161   # Google Repository, revision 32

        # Downloads the constraint-layouts files from Artifactory
        wget --user=${A_USER} --password=${A_PASS} https://artifactory.company.net/android-tmp/m2repository.tar.gz -O -| tar zfxv - -C "${GRADLE_USER_HOME}/android-sdk-linux/extras/"
        """

But when I run the build, I get the following error:

Entering stage Host preparation
Proceeding
[Pipeline] pwd
[Pipeline] withCredentials
[Pipeline] {
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
groovy.lang.MissingPropertyException: No such property: GRADLE_USER_HOME for class: groovy.lang.Binding
    at groovy.lang.Binding.getVariable(Binding.java:63)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:224)
    at org.kohsuke.groovy.sandbox.impl.Checker$4.call(Checker.java:241)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:238)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:221)

For debugging matters, I've created another build and wrote:

node {
    def cwd = pwd()
    stage 'itai'
    sh """
    echo '${cwd}'
    """
}

And then it works, the cwd variable gets populated with the current working directory. But in my own build it doesn't...

I'm using the pwd() function like that in another part of the pipeline and it works there so I don't understand why it doesn't work here, can you please check to see if you find the issue?

score 2 · Accepted Answer · answered Sep 15 '16 at 18:47

When you put $something or ${something} in your script, groovy will expand it to a variable...

So when you put: $GRADLE_USER_HOME, groovy is trying to expand that from a variable... But you don't have a GRADLE_USER_HOME variable in your groovy script...

What you are trying to do is output $GRADLE_USER_HOME into your shell script.

So you will need to escape the $ whenever it's not for Groovy to handle it...

ie:

    export NDK_DIR='\$GRADLE_USER_HOME/android-ndk-\$NDK_VER'

And all the other places you don't want groovy to handle the $

score 2 · Answer 2 · answered Sep 16 '16 at 12:03

tim_yates is right, you want to remember here that Groovy variables and shell variables both use the same "$" prefix, therefore you need to escape your shell variables so they do not get interpreted as Groovy variables.

However, in your case you you have another solution : you could also just define the shell variables you use... as Groovy variables and use them as Groovy variables, therefore avoiding escaping each and every "$" symbol.

Example :

stage 'Host preparation'
def cwd = pwd()
def ndkVer = "r12b"
def sdkVer = "r24.4.1"
def gradleUserHome = "${cwd}/.gradle"
def sdkTools="$gradleUserHome/android-sdk-linux/tools"
withCredentials([ // Use Jenkins credentials ID of artifactory
                  [$class: 'UsernamePasswordMultiBinding', credentialsId: artifactory_creds, usernameVariable: 'A_USER', passwordVariable: 'A_PASS'],
]){
    sh """
        export NDK_VER=$ndkVer
        export SDK_VER=$sdkVer
        export GRADLE_USER_HOME=$gradleUserHome
        echo "CWD OK"
        export NDK_DIR='$gradleUserHome/android-ndk-$ndkVer'
        export SDK_DIR='$gradleUserHome/android-sdk-linux'
        export PATH='\$PATH:$gradleUserHome:$gradleUserHome/android-ndk-$ndkVer:$gradleUserHome/android-sdk-linux'
        if [ ! -d $gradleUserHome ]; then
            mkdir $gradleUserHome
        fi

        if [ ! -f "$gradleUserHome/android-sdk-$sdkVer-linux.tgz" ]; then
            curl -o "$gradleUserHome/android-sdk-$sdkVer-linux.tgz" https://dl.google.com/android/android-sdk_$sdkVer-linux.tgz
        fi

        if [ ! -f "$gradleUserHome/android-ndk-$ndkVer-linux-x86_64.zip" ]; then # Checks if the sdk tarball exists on system
            curl -o "$gradleUserHome/android-ndk-$ndkVer-linux-x86_64.zip" https://dl.google.com/android/repository/android-ndk-$ndkVer-linux-x86_64.zip
        fi

        if [ ! -d "$gradleUserHome/android-ndk-$ndkVer" ]; then
            cd "$gradleUserHome" && unzip -o "android-ndk-$ndkVer-linux-x86_64.zip"
        fi
        if [ ! -d "$gradleUserHome/android-sdk-linux" ]; then
            tar -xzf "$gradleUserHome/android-sdk-$sdkVer-linux.tgz" -C $gradleUserHome
            mkdir "$gradleUserHome/android-sdk-linux/extras"
        fi

        echo "Installing updates"
        export SDK_TOOLS="${gradleUserHome}/android-sdk-linux/tools"
        # Downloads the required SDK tools
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 2     # Android SDK Tools, revision 25.2.2 rc1
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 3     # Android SDK Platform-tools, revision 24.0.2
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 4     # Android SDK Build-tools, revision 24.0.2
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 6     # Android SDK Build-tools, revision 24
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 7     # Android SDK Build-tools, revision 23.0.3
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 8     # Android SDK Build-tools, revision 23.0.2
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 10    # Android SDK Build-tools, revision 23 (Obsolete)
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 30    # SDK Platform Android 7.0, API 24, revision 2
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 31    # SDK Platform Android 6.0, API 23, revision 3
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 53    # Android TV Intel x86 Atom System Image, Android API 24, revision 6
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 54    # Android Wear ARM EABI v7a System Image, Android API 24, revision 1
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 55    # Android Wear Intel x86 Atom System Image, Android API 24, revision 1
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 57    # ARM EABI v7a System Image, Android API 24, revision 6
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 58    # Intel x86 Atom_64 System Image, Android API 24, revision 6
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 59    # Intel x86 Atom System Image, Android API 24, revision 6
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 153   # Android Support Repository, revision 36
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 160   # Google Play services, revision 32
        # echo "y" | "$sdkTools/android" update sdk -u -a -t 161   # Google Repository, revision 32

        # Downloads the constraint-layouts files from Artifactory
        wget --user=${A_USER} --password=${A_PASS} https://artifactory.company.net/android-tmp/m2repository.tar.gz -O -| tar zfxv - -C "$gradleUserHome/android-sdk-linux/extras/"
        """

Note that here I used camelCase notation to differentiate Groovy variables from shell variables, but of course you could use your upper case notation just the same. Also, you'll notice the PATH='\$PATH:$gradleUserHome... where the $PATH is escaped as \$PATH because this time it's a shell variable we're using...

Unable to use a variable within a "sh clause" in a Jenkins pipeline, what could be the reason?

2 Answers2