Streaming video from S3 with Express/NodeJS to play in Safari

Question

I am using SSE-C encryption for S3 files, so when playing back the video, I am forwarding the request to S3 through the server by attaching params correctly.

export default async function(req, res) {
  const {userid, fileid} = req.params;
  console.log(userid, fileid);
  const params = {
    Bucket: process.env.AWS_BUCKET,
    Key: `users/${userid}/notes/${fileid}`,
    SSECustomerKey: process.env.AWS_SSE_KEY,
    SSECustomerAlgorithm: "AES256"
  };

  s3.getObject(params)
    .on("httpHeaders", function(statusCode, headers) {
      console.log(headers);
      res.set("Content-Length", headers["content-length"]);
      res.set("Content-Type", headers["content-type"]);
      res.set("Accept-Ranges", "bytes");
      this.response.httpResponse.createUnbufferedStream().pipe(res);
    })
    .send();
}

While this works fine with HTML5 video tag in Chrome/Firefox, it doesn't work in Safari because return Content-Type is application/octet-stream (found a similar question Why won't Safari play file without extension in <video>?)

I have tried resetting the Content-Type manually to a correct video/quicktime or video/mp4 depending on a file, but that doesn't help.

I have also tried setting Content-Ranges manually, but that information doesn't get returned from S3 (or I am missing something), so that didn't help either.

Is this even possible to achieve? What am I missing?