How can I reference the Jenkinsfile directory, with Pipeline?

Question

I have a groovy file, I want to run from the Jenkinsfile.

ie. load script.groovy

However, I am not sure how I can reference this file if it is stored in the same directory as the Jenkinsfile. I am loading the Jenkinsfile from git. I noticed it creates a folder called workspace@script. It does not place this in the workspace directory. I could hardcode the folder but I am not sure the rules on this and it seems a little redundant to check-out the code again.

java.io.FileNotFoundException: /opt/jenkins_home/jobs/my_job/workspace/script.groovy (No such file or directory)

By default it loads from workspace, instead of workspace@script

I am trying to convert a a BuildFlow script to a Pipeline (Workflow) script. But I am finding, it is not as easy as a copy and paste.

Jenkinsfile

node {

//get parameters from Job
def builds = builds.tokenize(",")
def ip_address_node = ip_address_node.trim()
def port_node = port_node.trim()
def branch = branch.trim()
def workspace = pwd()

stage 'Checking out code from esb repository'
git branch: branch, url: 'ssh://git@giturl/integration_bus.git'

load '../workspace@script/esb_deploybar_pipeline/deploy_esb.groovy'

}

deploy_esb.groovy (this is from old buildflow, trying to run in a pipeline)

import groovy.transform.ToString
import groovy.transform.EqualsAndHashCode
@EqualsAndHashCode
@ToString
class BarDeploy {
    String barFile
    String app
    String integrationServer
}


//parse csv
def csvItemsApps = new HashSet<BarDeploy>();
def csvItemsLibs = new HashSet<BarDeploy>();
def deploymentMapFile = new File(workspace + "/ESB_Deployment_Map.csv")
def isFirstLine = true

stage 'Parsing ESB Deployment CSV'
deploymentMapFile.withReader { reader ->
    while(line = reader.readLine()) {
        if(isFirstLine)
        {
          isFirstLine = false
          continue
        }

        csvLine = line.split(",")
        app = csvLine[0]
        intServer = csvLine[1]

        def barDeploy = new BarDeploy()
        barDeploy.app = app
        barDeploy.integrationServer = intServer
        csvItemsApps.add(barDeploy)


        //get shared libs
        if(csvLine.length > 2 && csvLine[2] != null)
        {
            def sharedLibs = csvLine[2].split(";")
            sharedLibs.each { libString ->
                if(!libString.isAllWhitespace())
                {
                    def lib = new BarDeploy()
                    lib.app = libString
                    lib.integrationServer = intServer
                    csvItemsLibs.add(lib)
                }
            };
        }
    }
};

//get list of bar files to deploy from html and consolidate bar files to deploy with apps in csv 
for (int i = 0; i < builds.size(); i+=3)
{
    if(builds[i].equals("false"))
    {
        //Don't deploy bar if checkbox isn't selected
        continue
    }

    foundInCSV = false

    appToDeploy = builds[i + 1]
    barFileToDeploy = builds[i + 2]

    iterator = csvItemsApps.iterator()
    while (iterator.hasNext())
    {
        barDeploy = iterator.next()
        if(appToDeploy.equalsIgnoreCase(barDeploy.app))
        {
            barDeploy.barFile = barFileToDeploy
            foundInCSV = true
        }
    }

    iterator = csvItemsLibs.iterator()
    while (iterator.hasNext())
    {
        barDeploy = iterator.next()
        if(appToDeploy.equalsIgnoreCase(barDeploy.app))
        {
            barDeploy.barFile = barFileToDeploy
            foundInCSV = true
        }
    }

    if(foundInCSV == false)
    {
        throw new RuntimeException("App: " + appToDeploy + " not found in ESB_Deployment_Map.csv. Please add CSV Entry.")
    }
}


//Do deploy, deploy shared libs first
deployCSVItemsInParallel(ip_address_node,port_node,branch,env_key,csvItemsLibs)
deployCSVItemsInParallel(ip_address_node,port_node,branch,env_key,csvItemsApps)


def deploy(ip_address_node,port_node,branch,deployItem,env_key)
{
    def integrationServer = deployItem.integrationServer
    def app = deployItem.app
    def barFile = deployItem.barFile

    if(barFile == null)
    {
        return;
    }

    println("Triggering Build -> ESB App = " + app +  ", Branch = " 
            + branch + ", Barfile: " + barFile + ", Integration Server = " + integrationServer + ", IP Address: " + ip_address_node 
            + ", Port: " + port_node + ", Env_Key: " + env_key)

    build_closure = { ->
        build("esb_deploybar", 
                      ip_address_node: ip_address_node, port_node: port_node,
                      integrationServer: integrationServer, branch: branch, app: app, barFile: barFile, env_key: env_key)
    }

    return build_closure
}

def deployCSVItemsInParallel(ip_address_node,port_node,branch,env_key,csvItems)
{
    def build_closures = []
    iterator = csvItems.iterator()
    while (iterator.hasNext())
    {
      barDeploy = iterator.next()
      def build_closure = deploy(ip_address_node,port_node,branch,barDeploy,env_key)

      if(build_closure != null)
      {
          build_closures.add(build_closure)
      }
    }

    if(build_closures?.size() > 0)
    {
         parallel(build_closures)
    }
}

You should include as much code as necessary to show your problem, but as little code as possible, that keeps the code easy to understand. — Raúl Salinas-Monteagudo, Jul 01 '19 at 08:40
@CodyK would you consider marking my answer below as the correct one? — Mig82, Nov 18 '19 at 09:01

Mig82 · Answer 1 · 2017-02-09T14:14:20.750

There's one scenario that I haven't seen anyone mention. Which is how to load the Groovy scripts when the job is supposed to run on a Jenkins agent/slave, rather than on the master.

Since the master is the one that checks out the Jenkins pipeline project from SCM, the Groovy scripts are only found in the master's file system. So while this will work:

node {       
    def workspace = pwd() 
    def Bar = load "${workspace}@script/Bar.groovy"
    Bar.doSomething()
}

It's only a happy coincidence because the node that clones the pipeline from SCM is the same one that attempts to load the groovy scripts in it. However, just adding the name of a different agent to execute on:

node("agent1"){
    def workspace = pwd() 
    def Bar = load "${workspace}@script/Bar.groovy"
    Bar.doSomething()
}

Will fail, resulting in:

java.io.IOException: java.io.FileNotFoundException: /Jenkins/workspace/Foo_Job@script/Bar.groovy (No such file or directory)

This is because this path:

/Jenkins/workspace/Foo_Job@script/

Only exists on the master Jenkins box. Not in the box running agent1.

So if you're facing this issue, make sure to load the groovy scripts from the master into globally declared variables, so the agent can then use them:

def Bar
node {       
    def workspace = pwd() 
    if(isUnix()){
        Bar = load "${workspace}@script/Bar.groovy"
    }
    else{
        Bar = load("..\\workspace@script\\Bar.groovy")
    }
}
node("agent1"){
    Bar.doSomething()
}

Note: The variable used to pass the module between nodes must be declared outside the node blocks.

One gotcha here is if the number of executors on master node is set to zero. In that case it seems that `load` has no way of finding your groovy file. — Harold Putman, Jan 18 '17 at 21:46
You're right @HaroldPutman. In that case the only way I see forward would be to use a git step to clone the repository for the scripts (Jenkinsfile and Groovy) yet again from within the node("agent1") block. It's a waste to have to clone the scripts twice (From the master and then also from the agent), but you would be forced to it by an environment where no executors were allowed on master. — Mig82, Mar 02 '17 at 08:52

score 13 · Answer 2 · answered Jun 29 '16 at 19:44

13

If deploy_esb.groovy file is stored in the same SCM as the Jenkinsfile you can do:

node {       
   def workspace = pwd() 
   load "${workspace}@script/esb_deploybar_pipeline/deploy_esb.groovy"
}

answered Jun 29 '16 at 19:44

Anton Shishkin

1,696
1
8
10

8

This will fail if the job can be executed in parallel, as Jenkins will create workspaces like `jobname@2` and `jobname@3`, but reuse `jobname@script`. – kadrach Dec 02 '16 at 04:18

Mark Chorley · Answer 3 · 2016-05-25T15:49:12.690

1

if this script.groovy file is in the root of your project, like the Jenkinsfile, it will be fetched from git into the same folder as your Jenkinsfile. So the command you are using should work OK.

Are you getting some error? Please provide more details if so.

EDIT: now I can see what's in your Jenkinsfile, I can see you are checking out a git project called integration_bus which is where the groovy script resides. You can specify the location where that is checked out like this:

checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'esb_deploy']], submoduleCfg: [], userRemoteConfigs: [[url: 'ssh://git@giturl/integration_bus.git']]])

as opposed to what you have

git branch: branch, url: 'ssh://git@giturl/integration_bus.git'

Then you should be able to reference the groovy script in the esb_deploy folder like this

load 'esb_deploy/esb_deploybar_pipeline/deploy_esb.groovy'

edited May 25 '16 at 15:49

answered May 23 '16 at 15:35

Mark Chorley

2,087
2
22
29

The script loads from the workspace directory, instead of workspace@script, which isnt a big deal but id rather not hardcode this path. – CodyK May 24 '16 at 15:03
The groovy file is actually located in a different repository. It is located in same repository as Jenkinsfile. I got it to work by referencing file like this load '../workspace@script/esb_deploybar_pipeline/deploy_esb.groovy'. I am having other issues, but that those are different problems. Thanks – CodyK May 25 '16 at 17:45
This still feels a bit hacky, doesn't it? How about adding your groovy helper script to a git repo which you then add to the relevant projects as a submodule? – sebkraemer Nov 21 '17 at 09:54

amuniz · Answer 4 · 2016-05-31T15:36:56.577

0

You can just assume that all file operations in the Jenkinsfile are relative to the current workspace (which is the default workspace when using load inside a node).

So if the target file (say deploy_esb.groovy) is inside the folder foo in your SCM, this should work without additional configuration:

git branch: branch, url: 'ssh://git@giturl/integration_bus.git'
load 'foo/deploy_esb.groovy'

Or this if the file to load is in the same repository than the Jenkinsfile:

checkout scm
load 'foo/deploy_esb.groovy'

edited May 31 '16 at 15:36

answered May 26 '16 at 16:16

amuniz

3,292
2
20
22

The issue was, is the groovy file was located in the same repository as the jenkinsfile. Ignore that git clone, that isn't apart of it. I just included as I copied and pasted the jenkins file. But I load the jenkins file from SCM, so it creates a workspace@script directory. I just used this: '../workspace@script/deploy_esb.groovy' t load the script – CodyK May 27 '16 at 12:59
Then use: `checkout scm` followed by the `load` step. – amuniz May 31 '16 at 09:49
3

I know, It just seemed a little redundant to be checking out the code twice. – CodyK May 31 '16 at 14:51

score 0 · Answer 5 · answered Jul 17 '18 at 14:40

0

This should work

load "${WORKSPACE}/../${JOB_NAME}@script/esb_deploy/esb_deploybar_pipeline/deploy_esb.groovy"

answered Jul 17 '18 at 14:40

user10094410

1
1
1

This works for me with one issue: for jobs using folders, use JOB_BASE_NAME instead of JOB_NAME. – ZHANG Zikai Feb 15 '19 at 06:20

score -1 · Answer 6 · answered Dec 05 '18 at 13:47

Had the same problem. Ended up with extra clone operation to get copy of script repo under workspace dir so I can reliably access groovy files in it:

dir ('SCRIPTS_DIR') {
    checkout scm
    commonScripts = load 'subdir/Common.groovy' // no def, so script is global
}

How can I reference the Jenkinsfile directory, with Pipeline?

6 Answers6

Linked