gulp-foreach not looping over all files

Question

I have a very basic usage of gulp-foreach that's failing so badly I imagine I must be doing something wrong; I'm just (for now) trying to print all files found in gulp.src. When doing this with gulp-foreach only a very, very small sample of the files are ever printed—about 15 out of many hundreds. When using gulp-print things work as expected.

const gulp = require('gulp'),
      foreach = require('gulp-foreach'),
      gulpPrint = require('gulp-print');

gulp.src([`../../**/*.js`].concat([]))
    .pipe(gulpPrint(path => {
        console.log(path);
    }))
    .pipe(foreach((stream, file) => {
        console.log(file.path);
        return stream;
    }))

The code with the foreach is triggered about 15 times, interleaved with the gulpPrint, followed by a flood of gulpPrint statements alone. If I comment out the gulpPrint part, foreach prints about 15 times and then exits, and gulpPrint alone, as expected, floods the console with every file found, as it should.

I can just use gulp-print, but I'd like to know what I'm doing wrong with gulp-foreach.

score 2 · Accepted Answer · answered Aug 09 '16 at 18:49

First of all: gulp-foreach isn't really the right plugin for your use case.

gulp-foreach allows you to create a substream for each file and emit the results of the substream into the original stream. That means you can emit multiple files for each input file (see this example). In this regard it is very similar to the higher-order function flatmap in functional programming. Which is why the author of gulp-foreach has recently renamed the plugin to gulp-flatmap.

All you are interested in is to access each file in the stream and print its path, which makes this a case of the higher-order function map. The right plugins for this are map-stream or gulp-tap (which is what the author of gulp-foreach used to recommend).

Now to your actual question.

As already mentioned gulp-foreach emits all the files from the created substream to the original stream. However you are doing nothing with the files that are written to the original stream. Node.js object streams by default have a highWaterMark of 16, so as soon as gulp-foreach has written 16 files to the original stream the buffer is full and gulp-foreach blocks.

In order to fix this you need to consume the files that gulp-foreach writes to the stream. You could simply use a dummy .on('data') handler for this:

gulp.src([`../../**/*.js`].concat([]))
  .pipe(gulpPrint(path => {
    console.log(path);
  }))
  .pipe(foreach((stream, file) => {
    console.log(file.path);
    return stream;
  }))
  .on('data', (data) => { });

However the usual way to do this in gulp is to let gulp consume the stream instead by simply returning the stream from the task it is created in:

gulp.task('default', function() {
  return gulp.src([`../../**/*.js`].concat([]))
    .pipe(gulpPrint(path => {
       console.log(path);
    }))
    .pipe(foreach((stream, file) => {
       console.log(file.path);
       return stream;
    }));
});

Great answer - thank you. Is there any reason `gulpPrint` isn't as good an option as `gulp-tap`? — Adam Rackis, Aug 09 '16 at 19:16
For your use case: no, not really. `gulp-tap` is more general in that it gives you the entire [`vinyl` file](https://github.com/gulpjs/vinyl) which allows you to access and manipulate file contents etc. But if all you want to do is print file paths, `gulp-print` will do just fine. — Sven Schoenung, Aug 09 '16 at 19:34

score 2 · Answer 2 · answered Feb 13 '17 at 14:05

The reason why your are not seen all the files, is because you are not returning a promise, and the task is finishing before iterating over all the files

return the gulp.src like this:

const gulp = require('gulp'),
  foreac**strong text**h = require('gulp-foreach'),
  gulpPrint = require('gulp-print');

gulp.task('default', function () {
return gulp.src([`../../**/*.js`].concat([]))
    .pipe(gulpPrint(path => {
        console.log(path);
    }))
    .pipe(foreach((stream, file) => {
        console.log(file.path);
        return stream;
    }))

})

gulp-foreach not looping over all files

2 Answers2