How do I post a plain file buffer instead of binary-encoded-file when posting with nodejs/npm's request package?

Question

I am using npm's request package to do a post of a file-buffer to a REST api written using meteor.js restivus package. My node.js client code that posts to the api is as follows:

    url = 'http://localhost:3000/api/v1/images/';

fs.readFile('./Statement.odt', function read(err, data) {
    if (err) {
        throw err;
    }
    console.log(data);  //At this stage the file is still a buffer - which is correct
    var file = data;

    request.post({
      uri: url, 
      headers:   {
        'X-User-Id': userId,
        'X-Auth-Token': authToken
      },
      form: {
          file: file, //Inside the request.post the file is converted to binary encoding
          name:"Statement.odt"
      }
    }, function(err, httpResponse, body) {
      if (err) {
        return console.error('post failed:', err);
      }

      console.log('Get successful!  Server responded with:', body);
    });
});

The problem/issue here is that inside the request.post the file is converted to binary-encoded blob. See my comments in the "form:" property of the first argument of "request.post" in the code above. This becomes a problem on my meteor.js server where the file is required as a buffer instead of a binary encoded file. (For info: I am using Ostr-io/files' GridFS to store the file - it requires the file to be a buffer)

If there is no way other than to pass the file as an encoded string, then is there a way to convert that encoded blob back to a buffer server-side where I am using/talking meteor.js? Please help!

If you need more info, please let me know, I'll provide.

yusha uzumo · Accepted Answer · 2018-07-26T12:42:13.133

I found the answer to my own question. My node.js client code that posts to the api changes as follows:

 url = 'http://localhost:3000/api/v1/images/';
 fs.readFile('./Statement.odt', function read(err, data) {
  if (err) {
   throw err;
  }
  console.log(data);
  //var file = new Buffer(data).toString('base64');
  var file = data;
  //var file = fs.readFileSync('./Statement.odt',{ encoding: 'base64' });
  
  var req = request.post({
    uri: url, 
    headers:  {
   'X-User-Id': userId,
   'X-Auth-Token': authToken
    },
    /*form: { //Post to the body instead of the form
     file: file,
     name:"Statement.odt",
    },*/
    body: file, //Post to the body instead of the form
    json:true  //Set json to true
    //encoding: null
  }, function(err, httpResponse, body) {
    if (err) {
   return console.error('post failed:', err);
    }
  
    console.log('Get successful!  Server responded with:', body);
  });
 });

On the server-side access the json data as follows and convert the file back to a buffer:

var bufferOriginal = Buffer.from(this.request.body.data)

Note that when you do console.log(bufferOriginal) you get an output of the file encoded in utf8 but when you reference/use bufferOriginal anywhere in the code it is recognized as a file buffer that looks somenhing like:

<Buffer 50 4b 03 04 14 00 00 08 00 00 00 55 f6 4c 5e c6 32 0c 27 00 00 00 27 00 00 00 08 00 00 00 6d 69 6d 65 74 79 70 65 61 70 70 6c 69 63 61 74 69 6f 6e 2f ... >

Thanks to @Dr.Dimitru for nudging me in the direction towards the solution and for pointing out that this.request.body is already json

How do I post a plain file buffer instead of binary-encoded-file when posting with nodejs/npm's request package?

1 Answers1