71

What is the major difference between Job DSL Plugin and Pipeline Plugin

  1. both provide way to programmatic job creation
  2. which is the best to use as moving ahead and why?
  3. if both have similar functionality, do they have different use cases?
  4. Since Jenkins 2.0 is focusing on Pipelines as code, does this mean that job-dsl does not have a future or Pipeline Plugin is the next step in the Job DSL Plugin?
daspilker
  • 8,154
  • 1
  • 35
  • 49
Mr.Pramod Anarase
  • 1,454
  • 2
  • 15
  • 19
  • Read this https://jenkins.io/solutions/pipeline/ – CSchulz Jun 06 '16 at 13:35
  • 2
    They are complementary and pipeline may eventually replace Job DSL, but not yet. A good read is https://marcesher.com/2016/08/04/jenkins-as-code-comparing-job-dsl-and-pipelines/ – Felipe Jun 08 '17 at 13:21
  • Be careful with it https://issues.jenkins-ci.org/browse/JENKINS-33846 – Robinho Jan 23 '18 at 14:49

4 Answers4

83

I have extensive experience with both. A concise reply is that Job DSL has existed for much longer and was Netflix's open source solution for "coding" Jenkins. It allowed you to introduce logic and variables into scripting your Jenkins jobs and typically one would use these jobs to form some sort of "pipeline" for a particular project. This plugin received quite a bit of traction as a common way to enable job templating and scripting.

Jenkins Pipeline (2.0) is a new incarnation of a Jenkins job that is entirely based on a DSL and attempts to eliminate the need to stitch together multiple jobs to fill a single pipeline which was by far the most common use of Job DSL. Originally, with Pipeline DSL not offering many of the features that Job DSL did, and as mentioned above Job DSL would allow you to create Pipeline jobs, they could be used together to define a pipeline.

Today, IMO there is little reason to use Job DSL because Pipeline is the Jenkins-supported mechanism for scripting Jenkins pipelines and it has met or surpassed much of the functionality of Job DSL. New plugins are being developed natively for Pipeline, and those that don't are being encouraged by Jenkins developers to integrate with Pipeline. And Pipeline has several advantages:

  • There is no need to "seed" jobs using Pipeline as there is with Job DSL since the Pipeline is the job itself. With Job DSL, it's just a script that creates other jobs.
  • With Pipeline, you have features such as a parameterized manual input step, allowing you specify logic midstream within the pipeline
  • The logic that can be included in a Job DSL is limited to creating the jobs themselves; whereas with Pipeline you can include logic directly inside the job.
  • Job DSL is simply much more difficult to create a basic delivery pipeline using, for example, the Build Pipeline Plugin; using Pipeline your file will be smaller and syntax shorter. And if you're using Job DSL to create Pipeline jobs, I haven't seen a major value for that anymore given the templating features available out-of-the-box with Jenkins Pipeline.

Finally, Jenkins Pipeline is by far the most prevalent feature of Jenkins right now. Check out the Jenkins World 2016 agenda and you'll see approx. 50% of the sessions involve pipeline. None for Job DSL.

Neil
  • 2,524
  • 2
  • 22
  • 30
  • Great answer. One problem I had with my first attempt at the Jenkins 2.0 and the Pipeline plugin is that some of the plugins that I was using in build steps were not compatible. Im sure that will be changing in the near future – Robert Hutto Sep 07 '16 at 14:03
  • Yep, that is still the case, and certainly was even more so in early days. They are [starting to fill in](https://github.com/jenkinsci/pipeline-plugin/blob/master/COMPATIBILITY.md). – Neil Sep 08 '16 at 02:02
  • I think there is a typo, you wrote: "The logic that can be included in a Pipeline DSL is ..." whereas I think you meant "job DSL"? Maybe , to avoid confusing in other parts of your text, just talk about "Pipeline" instead of "Pipeline DSL" – Rabarberski Sep 15 '16 at 08:27
  • Thanks @Rabarberski, fixed it – Neil Sep 15 '16 at 13:24
  • 1
    The reason there was "none for Job DSL" is because the Job DSL plugin is far more mature and does out of the box most everything needed to create logic for a large number of jobs. Not because people aren't interested in it, it has a high number of uses and in many cases, the uses between both Pipeline and Job DSL can be combined for maximum effectiveness. – Highway of Life Mar 08 '17 at 02:00
  • Is it possible to use DSL features like managedScript in your pipeline? – lvthillo Nov 28 '17 at 08:36
  • 2
    How do you resume from a failed stage or repeat the deploy stage using Pipeline open source version? – Robinho Jan 23 '18 at 14:51
  • Please answer how do we resume from a failed stage or repeat the deploy stage using Pipeline open source version? @Robinho can you please advise – Ripunjay Godhani Jan 08 '19 at 08:25
  • @RipunjayGodhani you just need to define jobs and theirs dependencies. – Robinho Jan 19 '19 at 21:25
  • Yea, prevalent features, things that used to work with Job DSL, does not work with pipelines, e.g. https://stackoverflow.com/questions/66353946/jenkins-multibranch-pipeline-use-variable-in-project-repository, and they are not being fixed. – dzieciou Jan 26 '22 at 07:01
37

My feeling is the ideal approach is to use both. Pipeline is the new native Jenkins feature to have jobs as code. However, if building Jenkins from scratch, those jobs still need to be created. This means Jenkins can't be 100% truly scripted and built from code.

What you can do is use JOB DSL to build the skeleton structure of all the jobs, then use pipeline for the implementation of the jobs. This will allow you to 100% script Jenkins, minus the initial seed job to be created.

Maybe, eventually we will be able to use pipeline to fully control Jenkins (security, configuration and even plugins). But until then, I think it is a good approach to use DSL and Pipeline.

Wilfred Hughes
  • 29,846
  • 15
  • 139
  • 192
CodyK
  • 3,426
  • 4
  • 36
  • 52
  • 6
    This is very important to consider. You *cannot* create a fully automated Jenkins setup using only Pipeline. To support a fully automated Jenkins setup with Pipeline, you are (currently) required to use both the Netflix DSL and Pipeline (or create your pipeline jobs in groovy init scripts). I don't know if that will change in the future, but right now you cannot fully automate Jenkins with only Pipeline – Brandon Aug 01 '17 at 02:51
  • 2
    Exactly. This is what we are doing now. Using a combination of DSL and Pipeline. – CodyK Aug 01 '17 at 15:59
  • 2
    I think this is being worked on: https://github.com/jenkinsci/jep/tree/master/jep/201 – phant0m May 18 '18 at 11:47
4

There's one important distinction that's not been mentioned here so far: testing. With the job-dsl there's a means of testing the DSL code before committing it to SCM: https://github.com/sheehan/job-dsl-gradle-example - this allows a local test suite to be run on DSL code, as well as in Jenkins, before the code is run. As far as I'm aware, there is no equivalent in the native Pipeline approach.

Andrew
  • 580
  • 5
  • 13
  • 2
    The Jenkins Pipeline Linter Connector extension for VisualStudio Code takes the file that you have currently opened, pushes it to your Jenkins Server and displays the validation result in VS Code. (Jenkins server here can be on your local / test machine. ) https://jenkins.io/doc/book/pipeline/development/?source=post_page---------------------------#visualstudio-code-jenkins-pipeline-linter-connector Similar extensions are now also available for Atom & Sublime editors. – Swapnil jaiswal Jul 19 '19 at 06:09
3

My preliminary answer based on very limited experience:

  • Each uses a different Groovy DSL.
  • Job DSL gives you a way to create other jobs according to a Groovy script. So if you want a set of X related jobs (e.g. a pipeline), you'll create one Job DSL job, write the script, run that job, and then you'll have those X jobs, plus the one to create those jobs. At that point, only the job you created directly has been run, the ones created by that job have not. OOTB, these jobs are not hidden the way Maven modules in a multi-module Maven job are hidden, but I understand there is at least a way to create a view and stick the jobs there.
  • Pipeline DSL just hangs indefinitely in the 2 very different environments where I have tried it. :'( This is a known showstopper bug, as I understand--search and you will find some open bug tickets on this. Anywho, running the Pipeline job you create actually runs the pipeline, not generates a bunch of jobs like the Job DSL. So triggers to run the pipeline job are triggers to run the pipeline, not merely update the jobs it defines.
  • Pipeline appears to be more widely used, judging from the download numbers. Of course, Pipeline is a default feature of Jenkins 2.0, which might account for the recent spike in downloads. The maintainers of Jenkins sure want you to use it.

So to recap: Job DSL's DSL is for creating jobs that form a pipeline, Pipeline Plugin's DSL defines the pipeline itself.

And to answer your question(s): Pipeline should be more widely supported in the future, looks more straightforward to me (the job is a job, not a metajob), and appears to have more features (including workflow). I would use it unless you hit the aforementioned showstopper bug of doom and cannot find a fix / workaround.

Jason Young
  • 439
  • 1
  • 8
  • 18
  • 1
    I have no experience with the Pipeline plugin, but use Job DSL on a few projects. I wouldn't exactly say that Job DSL is for creating jobs *that form a pipeline*. It is for creating jobs, any jobs. By the way, I'm using it to generate similar (variability is addressed through input parameters) "freestyle" jobs that execute a .NET build. – odalet Jun 24 '16 at 19:15
  • 1
    You could even use Job DSL to generate Pipeline jobs. See http://stackoverflow.com/questions/35898020/job-dsl-to-create-pipeline-type-job – odalet Jun 24 '16 at 19:17
  • 1
    @odalet You are right on both comments. The Job DSL does not know whether you think of the jobs you create with it as part of one pipeline, although of course you would want to use a Job DSL job to create related jobs (whether or not they really form a pipeline). – Jason Young Jun 24 '16 at 21:39