Jenkins pipeline - How to iterate through a list

Question

I'm required to read values from a file in my pipeline. I'm using split() which puts them into an Array. I need to put them into an Arraylist so I'm using Arrays.asList(). The problem I'm having is I'm unable to use the size() or length() methods so I cannot make a for loop such as

for (ii = 0; ii < var.length; ii++)

or

for (ii = 0; ii < var.size; ii++)

because I get error: unclassified field java.util.Arrays$ArrayList length

So I tried to use a for each loop, but when I take some action (like ls command for example) in my finally block it only iterates 1 time. But if I just run the command 'echo' it iterates for each element like it's supposed to. Any advice on how to modify my code to get it to iterate for each element in the list when using any command?

Works correctly....

node{
    wrap([$class: 'ConfigFileBuildWrapper', managedFiles: [[fileId: 'dest_hosts.txt', targetLocation: '', variable: 'DEST_HOST']]]) {
        HOST = Arrays.asList(readFile(env.DEST_HOST).split("\\r?\\n"))
        deploy(HOST)
    }
}

@NonCPS
def deploy(host){
    for (String target : host){
        try {
            echo target
        }
        finally {
           echo target
        }
    }
}

OUTPUT (iterates for each element):

[Pipeline] node
Running on <obfuscated>
[Pipeline] {
[Pipeline] wrap
provisoning config files...
copy managed file [<obfuscated>] to file:/var/lib/jenkins/<obfuscated>
[Pipeline] {
[Pipeline] readFile
[Pipeline] echo
www.testhost.com
[Pipeline] echo
www.testhost.com
[Pipeline] echo
www.testhost2.com
[Pipeline] echo
www.testhost2.com
[Pipeline] }
Deleting 1 temporary files
[Pipeline] // wrap
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

But if I take some action such as 'ls -l' it only iterates 1 time

node{
    wrap([$class: 'ConfigFileBuildWrapper', managedFiles: [[fileId: 'dest_hosts.txt', targetLocation: '', variable: 'DEST_HOST']]]) {
        HOST = Arrays.asList(readFile(env.DEST_HOST).split("\\r?\\n"))
        deploy(HOST)
    }
}

@NonCPS
def deploy(host){
    for (String target : host){
        try {
            echo target
        }
        finally {
           sh 'ls -l'
        }
    }
}

OUTPUT (only iterates 1 time):

[Pipeline] node
Running on <obfuscated>
[Pipeline] {
[Pipeline] wrap
provisoning config files...
copy managed file [<obfuscated>] to file:/var/lib/jenkins/<obfuscated>
[Pipeline] {
[Pipeline] readFile
[Pipeline] echo
www.testhost.com
[Pipeline] sh
[sandbox%2Fpipeline-test-new1] Running shell script
+ ls -l
total 8
-rw-r--r-- 1 jenkins jenkins 10 Jun 17 16:07 someFile
[Pipeline] }
Deleting 1 temporary files
[Pipeline] // wrap
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

score 11 · Answer 1 · answered Jul 08 '16 at 05:09

11

ArrayList (and generally Lists) don't have a length or size field, they have a size() method. So use that in the for:

for (ii = 0; ii < var.size(); ii++)

answered Jul 08 '16 at 05:09

Krzysztof Krasoń

26,515
16
89
115

1

Thanks that works. Although I'm still curious why the 2nd for loop above only iterates one time, any ideas? – mdo123 Jul 08 '16 at 23:32
I'm now experiencing something similar. As far as I can tell, it is related to the `sh` call. Did you figure out something about this? – pmmaga Dec 22 '16 at 16:25
@pmmaga no sorry I didn't think about that. I just used @ krzyk's answer but I've unaccepted @ krzyk's answer because I think part of answer should include explanation on behavior I described above. Specifically, why does the for each loop work with "echo" in the finally block but only iterates once when sh command is executed. I thought maybe the sh cmd was returning a non zero exit status, but the for each loop should continue for each element, so I'm not sure. – mdo123 Jan 02 '17 at 18:32
@krzyk can you please update your answer to include explanation on why the for each loop works for the echo command but not the sh command? – mdo123 Jan 02 '17 at 18:33

score 6 · Answer 2 · answered Jan 31 '19 at 12:00

6

I prefer this solution:

node('master') {
    stage('Test 1: loop of echo statements') {
        echo_all(abcs)
    }
}

@NonCPS // has to be NonCPS or the build breaks on the call to .each
def echo_all(list) {
    list.each { item ->
        echo "Hello ${item}"
    }
}

If you use a declarative pipeline, you have to wrap the call in a script statement:

stage('master') {
    steps {
        script {
            echo_all(abcs);
        }
    }

answered Jan 31 '19 at 12:00

Matthias M

12,906
17
87
116

seems legit. It would be helpful to note why you prefer that as your solution and maybe give a link to what a declarative pipeline is, source ~ https://jenkins.io/doc/book/pipeline/syntax/#declarative-pipeline – mdo123 Jan 31 '19 at 17:09

Olaf · Answer 3 · 2017-08-25T12:53:39.290

As per this tutorial: https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md#serializing-local-variables

...a method marked with the annotation @NonCPS... will be treated as “native” by the Pipeline engine, and its local variables never saved. However it may not make any calls to Pipeline steps

In your case, the sh call is a pipeline step operation, which you apparently can't perform from within a @NonCPS annotated method.

Regarding turning an array into a List, then since we're in Groovy land you could just use the .toList() method on the array.

RustyCar · Answer 4 · 2017-08-07T17:27:42.580

I cannot tell you PRECISELY why, as I've not figured out how to find useful information about Jenkins without spending hours googling, but I can tell you this:

For a moment I thought you can make it run fine by adding 'echo line' AFTER the sh 'echo $line', but that turned out to be caused by Jenkins running a PREVIOUS version of the script...

I tried all sorts of things and none of them worked, then I found this:

Why an each loop in a Jenkinsfile stops at first iteration

Its a known bug in Jenkins pipeline!

(the known bug is JENKINS-26481, which says "At least some closures are executed only once inside of Groovy CPS DSL scripts managed by the workflow plugin")

Jenkins pipeline - How to iterate through a list

4 Answers4