1

i would like to limit the request size for an upload in the following case (simplified):

app = Express();
router = Express.Router();
router.use('/upload', (req, res, next) => {
  const writeStream = fs.createWriteStream('filename');
  req.pipe(writeStream);
});
app.use(router);

As you can see, no form or similar is used (no way to change this). Therefore, the data is provided as a raw data stream.

As I could not find any existing modules for this case (body-parser does not work for this case, raw-body can also not be used as the stream should be left untouched), my current approach would be to use the following code as a middleware for express:

function(req, res, next) {
  if (req.headers['content-length'] > limit) {
    res.set("Connection", "close");
    res.status(413).end();
    return;
  }
  let received = 0;
  req.on('data', (chunk) => {
    received += chunk.length;
    if (received > limit) {
      res.set("Connection", "close");
      res.status(413).end();
    }
  })
  next();
})

Is there any better way to do this? Is there any existing npm package that can do this job?

Philip
  • 135
  • 1
  • 2
  • 8
  • If you are using `multer` to upload the file it has an option to set the limits `limits: { fileSize: maxSize }` Or else [this](https://stackoverflow.com/questions/9049993/node-js-how-to-limit-the-http-request-size-and-upload-file-size) may help. – ambianBeing Sep 24 '19 at 14:08
  • Thanks, but as I wrote above the data is not provided as form / multipart data but as raw data. Therefore I can not use multer and I tried all of the steps in the provided link but none seems to work. – Philip Sep 24 '19 at 14:11
  • 1
    You said raw body should be untouched right. `body-parser` does indeed has an option to do just that `bodyParser.raw(options)` [source](https://github.com/expressjs/body-parser/blob/master/README.md#bodyparserrawoptions) where one of options is to set limit/size. – ambianBeing Sep 24 '19 at 14:26
  • Okay I already saw that, but as far as I know the raw bodyparser reads the full stream ("A new body object containing the parsed data is populated on the request object after the middleware (i.e. req.body)."). Therefore, the stream can no longer be used afterwards. – Philip Sep 24 '19 at 14:52
  • Yep! it'll do exactly that. Maybe I didn't get the requirement correctly. Not sure if there's another way to get the size apart from writing your own middleware to compute size or the way you're relying on `content-length header (both here). Also `buffers` can be converted into `streams` but that defeats the purpose here i guess. – ambianBeing Sep 24 '19 at 16:07
  • app.use(bodyParser.urlencoded({ extended: true, limit: "50mb" })); app.use(bodyParser.json({ limit: "50mb" })); – moein rahimi Sep 24 '19 at 16:16

1 Answers1

0

In the end I solved it by using the answer provided by @ambianBeing and rewrote the function that afterwards handles the buffer instead of a stream.

Philip
  • 135
  • 1
  • 2
  • 8
  • Even if you convert the buffer into a stream, body-parser will still read the entirety of the data into memory as a buffer. So if your data is large, it will all be read into memory first before you get the stream to compute the file size, which defeats your purpose in limiting the size of the upload in the first place? – tsaixingwei Sep 14 '22 at 02:11