Sharp unable to read file buffer

Question

I'm using express-fileupload for reading files from the API. Now I want to process the image in the request body using Sharp.

I don't want to first save the file at the server and process it using fs.readFileSync.

I tried passing req.files.image.data which is supposed to be a buffer.

  const image = await sharp(Buffer.from(req.files.image.data))
    .resize(500, 500)
    .jpeg({ quality: 10 })
    .toBuffer()
    .then((outputBuffer) =>
      ({ data: outputBuffer, mimetype: 'image/jpeg' }))
    .catch(err => {
      console.log(err);
      return null;
    });

But it is throwing error this error: [Error: VipsJpeg: Premature end of input file]

When I tried converting the image buffer data into string as suggested in this post, converting it into buffer using Buffer.from and then passing it, it throwing error: [Error: Input buffer contains unsupported image format]

Edit: There was a limit on image size 5mb, that's why the images greater than that were not getting completely captured in the buffer, hence, this error.

app.use(fileUpload({
  limits: { fileSize: 50 * 1024 * 1024 },
}));

score 4 · Accepted Answer · answered Jul 14 '22 at 12:15

Some questions arise, when I see your code. Let me try to get closer to a possible solution with your provided input by asking some questions and providing hints:

Are you sure your input file is not a corrupted format? It does not meet the requirements of jpeg specs probably? If you can be sure that the format is correct, try the next steps....
If req.files.image.data is really a buffer, why do you try to generate a buffer again by using Buffer.from(req.files.image.data)? You want to create a buffer from a buffer?
By the way - I line 4 you again try to conduct a conversion to a buffer with .toBuffer(). From an already existing buffer? In this case, I speak from personal experience, sharp would throw an error if trying to create a buffer from an already existing buffer.
You mention req.files.image.data is supposed to be a buffer. Sounds maybe you are not 100% sure. I suggest to check if you really have a buffer by using
const isItReallyBuffer = Buffer.isBuffer(req.files.image.data) After that you can simply print it to the console: console.log(isItReallyBuffer); // true or false
In the end, it shouldn't make a big difference if a string, a buffer or certain kinds of arrays are provided to sharp as input. As per documentation, Sharp is very flexible when it comes to input data and accepts Buffer, Uint8Array, Uint8ClampedArray, Int8Array, Uint16Array, Int16Array, Uint32Array, Int32Array, Float32Array, Float64Array and as mentioned a string.

Maybe check the type of your provided input req.files.image.data one more time. Is it really a not corrupted image file? Does it really comply with the input options accepted by Sharp listed above?
If yes, rather try const image = await sharp(req.files.image.data)...
If you already use a buffer, remove .toBuffer()

AMENDMENT fs.readFileSync is often used to process image files. In this case, I speak from my personal experience of many days of working with image files in node.js, I would think about using fs and better prefer the Sharp package for reading and writing image files. I don't use fs anymore. Fs is concatenating image chunks which in turn increases the probability leading to memory hogs.

You can simply open an PNG image on your desktop with WordPad or Notepad and search for IDAT chunks. Then, process the same image with fs package and you will see the difference; all of a sudden you likely have only one very huge IDAT chunk.

Sharp unable to read file buffer

1 Answers1