Prevent express body-parser from removing body prior to request.pipe

Question

I am trying to create a simple express middleware filter which should look at the POST body and determine if it should be piped to the correct server, or blocked. If I use body-parser it appears that I do get the req.body available to me, but it disappears from the request when I pipe the request to another external server.

The way I have tried to solve this is the following:

//Router
router.route('')
    .post(authController.isAuthenticated,
        urlHelper.bodyParser,             //Without these two, everything pipes correctly
        urlHelper.filterBasedOnSomething, //Without these two, everything pipes correctly
        urlHelper.pipeToTrustedZone);

In urlHelper:

const filterBasedOnSomething = function (req, res, next) {
        if (req.body
            && req.body.something === 'some string'
            && req.body.somethingElse === 'some other value') {
            return res.status(403).send("You shall not pass");
        } else {
            next();
        }
    }
const pipeToTrustedZone = function (req, res) {
        //Here we are figuring out the correct forwarding url. This is simplified a little for the question, but it is well tested and works.
        var url = getProxiedPath(req.baseUrl) + req.originalUrl;
        req.pipe(request({ qs: req.query, uri: url }).on('error', function (err) {
            log.info(err);
            return res.sendStatus(400);
        }))
            .pipe(res);

};
module.exports = {
  bodyParser: require('body-parser').json(),
  filterBasedOnSomething: filterBasedOnSomething,
  pipeToTrustedZone: pipeToTrustedZone,
  //..
}

This seem to give me the req.body in my filter-method, but the body is consumed and is not received after it has been piped forward. I have tried multiple things, like req.emit('data', JSON.stringify(req.body)); and req.write(.., but it all seem to fail.

Is there a way for me to look at the request body before piping it further without deleting it? Or is there an inherit problem with my approach?

I have found multiple github issues and SO questions relating to this, but I haven't been successful in getting any of those approaches to work.:

https://github.com/expressjs/body-parser/issues/74

Writing express middleware to get raw request body before body-parser

https://github.com/expressjs/body-parser/issues/94

rsp · Accepted Answer · 2017-06-06T15:24:48.863

1

Is there a way for me to look at the request body before piping it further without deleting it? Or is there an inherit problem with my approach?

The inherent problem is that you want to read a stream first and then pipe it - even though the stream has already been read and it's gone.

What you need to do is to either cache the stream while it is read so that it can be piped later or to reconstruct the body from the output of body-parser.

But you cannot rewind a stream and start reading it once again because it would mean recording all of the stream events in memory and the usual advantage of using streams is that you don't need to record everything in memory, you just process one data event at a time.

If the decision of whether or not you want to pipe the stream can be based on the data outside of the body - like a path or some headers etc. - then you can use the body-parser middleware only for those cases that are not piped.

But if the decision is based on the actual contents of the body as is the case in your example then you have no choice but to read it first - at which point it cannot be read again.

There are few modules that could help you with that:

but you may be better off by reconstructing the body from the parsed output of body-parser with something like:

request({ qs: req.query, uri: url, body: req.body, json: true })

and piping only the response.

edited Jun 06 '17 at 15:24

answered Jun 06 '17 at 15:14

rsp

107,747
29
201
177

I see. Do you know how I could record the data and re-resend it later? I have tried `req.emit('data', bodyData);` with and without `req.emit('end');`. I have tried a whole bunch of things. Maybe there is a way to re-create the `req` variable and pipet that further? – Automatico Jun 06 '17 at 15:22
1

@Cort3z If you don't need to pipe the exact same stream but just the same data then you may be able to add your body to the request. See the updated answer. – rsp Jun 06 '17 at 15:29
I am getting `{"message":"write after end","name":"Error","stack":"Error: write after end\n at ClientRequest.write (_http_outgoing.js:485:15)\n` when setting `body: req.body`. Does it need to be a stringified json? – Automatico Jun 06 '17 at 15:36
1

@Cort3z IT seems like you're trying to write to the stream returned from the `request()` - if you put the body in the request then you shouldn't pipe the request using `req.pipe()` or anything else, or use `.write()` on the stream returned from `request()` - just `request({ qs: req.query, uri: url, body: req.body, json: true })` and then only read from the returned stream (or pipe it to your `res` stream - but not pipe or write anything to it). – rsp Jun 06 '17 at 16:00
I see. What I ended up doing was straight up making a new request, and transfering the status code and data back into the original `res`, which seem to work nicely. Thanks for the help! – Automatico Jun 06 '17 at 19:13
@Automatico I'm also getting "write after end" error. Can you please tell me how did you pipe the post request? – Manisha Oct 30 '17 at 12:19
@Manisha I consumed the original request and made a new one on demand. – Automatico Oct 30 '17 at 22:31

Prevent express body-parser from removing body prior to request.pipe

1 Answers1