How can I use "express-http-proxy" after bodyParser.json() has been called?

Question

I am building a cross system admin app, which will be used as an admin tool for multiple backend systems. The app is built on top of Mean.js.

I have setup a /proxy route using "express-http-proxy" to send all sub-routes to their respective backend system endpoints. However, I need to have each request authenticated within my admin app and then decorated with the targeted backendSystem credentials before the "express-http-proxy" can continue. Here's an example of my /proxy route...

app.use('/proxy', users.requiresLogin, expressHttpProxy(config.backendSystem.host, {
    forwardPath: function (req) {
        return '/1.0' + require('url').parse(req.url).path;
    },
    decorateRequest: function (req) {
        req.headers['content-type'] = 'application/json';
        req.headers['backend-system-id'] = config.backendSystem.id;
        req.headers['backend-system-key'] = config.backendSystem.key;
        return req;
    }
}));

NOTE:
Currently the backendSystem credentials are stored based on the environment my admin app is ran in. However, in the future the backendSystem credentials will be specified by the user, and this /proxy route will differently than what is currently shown.

THE ISSUE:
Proxy routes that require data within the request body don't work.
e.g. POST /comments {"user": user_id, "text": "rabble rabble rabble"}

WHAT I'VE FOUND:
bodyParser.json() and "express-https-proxy" don't play nice. I've confirmed this by removing bodyParser.json() from express.js. However, this isn't a full solution since almost all of my other routes need bodyParser.json, e.g. /auth/signin.

Does anyone have a clean way that I can make a route exception for my /proxy route so that bodyParser.json won't be called for it?

score 6 · Answer 1 · answered Jul 16 '15 at 12:07

As far as I understand, the root of problem is so:

if you were reading a POST request by pure node, you should be using a code like this

if (req.method == 'POST') {
    console.log("POST");
    var body = '';
    req.on('data', function (data) {
        body += data;
        console.log("Partial body: " + body);
    });
    req.on('end', function () {
        console.log("Body: " + body);
    });
    res.writeHead(200, {'Content-Type': 'text/html'});
    res.end('post received');
}

in other words, you need to use the req.on('data') & req.on('end') events. but the problem is,that you can use this code only once. after the 'end' is called, the request is consumed.

so then you use bodyParser , it consumes the request, and the proxy have nothing to do with it.

actually, in my opinion, the proxy wait for the 'data' event to appear , but it will newer happen, so the code halts.

The solution:

you need to 're-enable' the events. I used this code and it works for me.

var express = require('express');
var bodyParser = require('body-parser');
var http = require('http');

//call for proxy package
var devRest = require('dev-rest-proxy');

//init express (as default)
var users = require('./routes/users');
var app = express();
app.use(bodyParser.json());

//set the proxy listening port
app.set('port', 8080);

//process the POST request
app.post('/users/*', function(req, res) {

    //just print the body. do some logic with it
    console.log("req.body: ",req.body);

    //remove listeners set by bodyParser
    req.removeAllListeners('data');
    req.removeAllListeners('end');

    //add new listeners for the proxy to use
    process.nextTick(function () {
    if(req.body) {
       req.emit('data', JSON.stringify(req.body));
    }
    req.emit('end');
    });

    //forward the request to another server
    devRest.proxy(req,res, 'localhost', 3000);    

});

//start the proxy server
http.createServer(app).listen(app.get('port'), function(){
    console.log("Express server listening on port " + app.get('port'));
});

module.exports = app;

the solution found on schumacher-m post (github of nodejitsu)

i only get part of the body when using this. so works with smaller payloads — Aryeh Armon, Apr 04 '18 at 14:49

score 1 · Answer 2 · edited May 23 '17 at 12:25

1

I was able to resolve my issue by adding a regex that excluded my /proxy route to where bodyParser.json was being added within express.js. I found that from this answer

While this approach doesn't scale well, it solved my immediate issue.

edited May 23 '17 at 12:25

Community

1
1

answered Feb 06 '15 at 19:17

hypno7oad

1,441
1
19
28

1

Can you explain your solution ? I am facing the same issue – cucucool Feb 10 '16 at 02:44

score 0 · Answer 3 · answered Oct 10 '18 at 12:20

I get it works by converting the data into query string using 3rd party query-string as follows:

proxyReqBodyDecorator: function(bodyContent, srcReq) {
    return (queryString.stringify(bodyContent));
}

Have tried JSON.stringify but not working, need the data in the following format array_field=val1&array_field=val2&array_field=val3......

score 0 · Answer 4 · edited Sep 05 '20 at 20:36

To modify the request body, do this with the latest express-http-proxy v1.6.2:

const express = require('express');
const proxy = require('express-http-proxy');
const bodyParser = require('body-parser');

const conf = {
    proxyHost:      'some.example.net:9200',
    proxyOptions: {
        proxyReqBodyDecorator:  modifyRequestBody,
        preserveHostHdr:        true,
        parseReqBody:           true
    },
    port:   8073
};

var app = express();
app.use('/proxy', proxy(conf.proxyHost, conf.proxyOptions));

function modifyRequestBody(body, srcReq) {
    if(srcReq.method.match(/^(GET|POST)$/i)) {
        try {
            // convert buffer to string, then to object
            var str = Buffer.from(body).toString('utf-8');
            var reqBody = JSON.parse(str);
            if(someCondition)) {
                reqBody.addStuff = 'whatever';
                body = reqBody; // return modified body as object
            }
        } catch(error) {
            console.log('- error: ' + JSON.stringify(error));
        }
    }
    return body; // return original buffer, or modified object
}

app.listen(conf.port, function () {
    log('app listening on port ' + conf.port);
});

score -1 · Answer 5 · answered Aug 25 '16 at 10:54

You can fill the proxyReq.bodyContent inside the decorateRequest method with the JSON-ed data from originalReq.body to be correctly POST'ed:

app.use('/proxy', users.requiresLogin, expressHttpProxy(config.backendSystem.host, {
  ...
  ...
  decorateRequest: function (proxyReq, originalReq) {
    ...
    ...

    if (originalReq.body) {
      proxyReq.bodyContent = JSON.stringify(originalReq.body);
    }
    return proxyReq;
  }
  ...
  ...
}));

it dows not work. There is no such variable in request. – Mindaugas Jaraminas Feb 17 '17 at 08:35 — Mindaugas Jaraminas, Feb 17 '17 at 08:35

How can I use "express-http-proxy" after bodyParser.json() has been called?

5 Answers5