1

I'm reading a file using Node and attempting to send it as a multi part MIME post, but having problems with the file appearing corrupt at the receiving end.

I read the file and prepare it for sending like so:

fs.readFile("before.png", function(err,data) {
    // Snip
    content += data.slice(0,data.length);

The problem is that something seems to padding the stream. See images below comparing the before.png source file with the file as received at the destination. The sequences of EF BF BD indicate that .slice() is not giving me the underlying bytes, possibly something coerced to UTF-8 encoding.

enter image description here

I've also tried getting the bytes via toString(), but no beans there. I still see corruption in the uploaded file. 

 // content += data.toString() // UTF-8 default
 content += data.toString('binary')

I suspect the default toString() is also coercing the binary file to UTF-8 too, but would have expected 'binary' to give me the underlying byte stream?

Edit in response to Brad. I tried concattin'g but I still need to translate my object back to a string, at which point the UTF-8 characters seem to reappear in the stream. 

contentToSend = Buffer.concat([ header, data, footer ] );
this.oauth.post( endpoint, accessToken, accessTokenSecret, contentToSend.toString(), contentType, function( x, y, z ) {
Benjamin Wootton
  • 2,089
  • 20
  • 21

1 Answers1

3

What I think is happening is that you are actually using a concatenation operator, forcing Node.js to treat content and data as strings with the default encoding of UTF-8. Try this instead:

content = Buffer.concat(content, data);

Also, there is no reason to get a slice for the full length of data. You can use the whole incoming buffer as-is.

Brad
  • 159,648
  • 54
  • 349
  • 530
  • Thanks very much for the creative suggestion. At some point I do need to convert it back to a String to send in my HTTP post. Even after concatt'ing and toString() the whole thing I see the same corruption. Will attach the code I am now using to the post above so I can format it. – Benjamin Wootton Jul 28 '13 at 16:30
  • TBC - I wonder if I'm suffering from the same as this guy - http://stackoverflow.com/questions/14855015/getting-binary-content-in-node-js-using-request – Benjamin Wootton Jul 28 '13 at 16:44
  • @BenjaminWootton, Your problem then has nothing to do with your buffer then, but with the source data encoding. Use whatever encoding the file is in, and your problem should go away. If you want something where what comes in is as what goes out, then you will have to treat the file as binary. You could try that guy's `none` encoding first though. – Brad Jul 28 '13 at 17:04