Does gulp.src().pipe().dest() complete before the next line? Can I call gulp.src().pipe().dest() multiple times?

Question

I'm supporting a build process that uses gulp and trying to troubleshoot a problem. To troubleshoot this, I'd like to know if the code that makes multiple calls to gulp.src().pipe().dest() in a single gulp task is legal and correctly written.

More exactly, I want to know if gulp.src().pipe().dest() completes prior to the next javascript line that makes another call to gulp.src().pipe().dest().

The code would look like this:

gulp.src(getSrcFiles())
    .pipe(dostuff)
    .pipe(dostuff2)
    .dest('temp');
// will the above copy complete before I try doing substitutions 
gulp.src('temp/**')
    .pipe(mySubstitutions)
    .dest('temp')
gulp.src('temp/**')
    .dest('dist')

The question is will the first pipeline of gulp.src().pipe().pipe().dest() complete prior to the next line which performs some variable substitutions on the files copied in the first pipeline?

I didn't see the answer in the gulp documentation at https://gulpjs.org/API.html#gulp-dest-path-options.

Some other questions I found that did not answer my question were:

• One Promise for Multiple Promises - Concurrency Issue

• Q.js Promises in Gulp Loop (not answered, but similar issue being raised).

• Using Gulp 4 with Promises to Clean/Copy Files

• gulp run sequence after promise resolve (similar question not answered)

• How does gulp treat the Promise of a then()? (similar question, crickets, no answer:-))

More questions but no answers... I have a gut feeling I'm going to answer this one myself after I figure it out, and then I'll be able to answer some of the other questions.

A great question and answer that will help me understand gulp and how it ensures ordering of tasks is What does Gulp "done" method do?. The gulp npm run-sequence docs were also very useful, see https://www.npmjs.com/package/run-sequence.

UPDATE: I found someone else who reported this as a problem.

• See https://github.com/gulpjs/gulp/issues/1254

Answer 1

This is the answer I was looking for:

• Will gulp.src().pipe().dest() completes prior to the next javascript line that makes another call to gulp.src().pipe().dest(). No. A stream pipeline is started but not necessarily completed when the next javascript line is run.

• Can you use multiple gulp.src().pipe().dest() in a single gulp task? Yes, but you must write the gulp task in a special way to handle this (shown below).

The following code shows how to write a gulp task that makes several gulp.src().dest() copies. The way it works is to convert the gulp.src().pipe() stream into a promise, and then use Promise.all() to wait for all of the streams to complete before saying that the gulp task is complete.

gulp.task('task', function (doneWithTaskCb) {
  return Promise.all([
    new Promise(function(resolve, reject) {
      gulp.src(src + '/*.md')
        .pipe(plugin())
        .on('error', reject)
        .pipe(gulp.dest(dist))
        .on('end', resolve)
    }),
    new Promise(function(resolve, reject) {
      gulp.src(src + '/*.md')
        .pipe(plugin())
        .on('error', reject)
        .pipe(gulp.dest(dist))
        .on('end', resolve)
    })
  ]).then(function () {
    doneWithTaskCb();
  });
});

I didn't come up with it. The credit goes to TySound at https://github.com/gulpjs/gulp/issues/1181#issuecomment-126694791.

The important code is this pattern:

new Promise(function(resolve, reject)  {
    gulp.src()
      .pipe()
      .pipe(gulp.dest(dist))
      .end('end', resolve);
}

That code creates a promise that when it resolves the pipeline is finished. So in essence, it converts the "stream" into a promise.

The code that calls doneWithTaskCb() is important because that is how gulp knows that the current task is complete.

Promise.all( [ new Promise(), new Promise() ] )
    .then(function() { 
        doneWithTaskCb(); 
    });