0

I'm trying to get JSON data from a data provider with terrible, terrible documentation. Their API's response has a content-type of 'application/octet-stream' which, if I use my standard API consumption approach, ends-up serving me with a file containing the data... and setting Accept to 'application/json' in my Request does not make any difference. So I'm trying to transform the octet-stream in JSON on my side and this is what I managed to do:

const request = require("request");
module.exports.test = function() {
  var text = "";
  var url = "XXX";
  request
    .get({
      url: url,
      headers: {
        Accept: "application/json"
      }
    })
    .on("response", function(response) {
      response.on("data", function(data) {
        text = text + data.toString();
      });
    })
    .on("end", function() {
      console.log(text); //This is where I stopped...
    });
};

That's where I stopped, the JSON is printed in the log and so I could go on with my life and use the amazingly well named variable 'text' to do what I need to do with the data... BUT:

  1. Is this the best way for dealing with octet-stream responses?
  2. Should I trust a data provider (and pay for their data) that is not able to provide an API with responses other than application/octet-stream ?
BoDeX
  • 846
  • 8
  • 18
  • check [this answer](https://stackoverflow.com/questions/17836438/getting-binary-content-in-node-js-with-http-request), then you can do simply do: `let json = JSON.stringify(yourBuffer);` – Margon Mar 05 '19 at 11:19
  • thanks @Margon what you are saying is adding the data to an array is better, I'm guessing for performance reasons? But is the order in the array guaranteed? I wouldn't want my data to be scrambled... Other than that it seems to be pretty similar to my solution (separate from my use of the request package), except for the specifying of 'base64' in the toString()... – BoDeX Mar 05 '19 at 11:46
  • Arrays mantains the order, don't worry, your data will be transformed to an array object (if the data are an array). For the performance issues, it depends... If there are a huge amount of data I would suggest to handle them in the server. the transformation in base64 is a specific thing for binary data (example a whole file retrieved from a server). If you are sure that your data are actually JSON you can just do the "data.toString()" and it should works. Otherwise use the transformation to base64 :) – Margon Mar 05 '19 at 11:52
  • I see. The data should always be JSON. I didn't mention it but this is already server side (Node.js). The server gets the data from the provider and stores it for later use. – BoDeX Mar 05 '19 at 11:58

0 Answers0