Creating buffer object from fetch response Javascript/Nodejs

Question

I am sending an API request using fetch and returning back JSON response. I am having a hard time at creating a buffer object so I can manipulate/parse through the JSON response like I would if it was a local file or variable containing JSON object. Below is my fetch code (which works):

const fetch = require("node-fetch");


const asset = 'Build_Artifacts'
const url2 = 'http://localhost:8081/service/rest/v1/components?repository='
const myHeaders = {
    "Authorization": "Basic token",
    "Content-Type": "application/json",
};

const requestOptions = {
    method: 'GET',
    headers: myHeaders,
    redirect: 'follow'
}

const getData = async () => {
    const res = await fetch(url2 + asset, requestOptions);
    const data = await res.text()
    return data
}

getData()   // Returns JSON response

From reading: https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Client-side_web_APIs/Fetching_data How do I read the contents of a Node.js stream into a string variable? https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch

I gather I should be creating a buffer object to throw the fetch response into. Once I have done this, then I should be able to proceed with manipulating the response content. I have made a few newb attempts and realize I am no where close. Below is a newb attempt based off the above reading:

getData().then(data.arrayBuffer().then(function(buffer) {
    //do something with the buffered object
}))

Update I am using res.text() instead of res.json() because my fetch response when using res.json() does not come back as verified JSON output. See below:

//Output when using res.text():
{
  "items" : [ {
    "id" : "T0lDX0J1aWxkX0FydGlmYWN0czpmODk4YjM5MDNjYjk5YzU5NjgzNTk3ZWRjNWFmZjliYw",
    "repository" : "Build_Artifacts",
    "format" : "raw",
    "group" : "/",
    "name" : "05baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36.zip",
    "version" : null,
    "assets" : [ {
      "downloadUrl" : "http://localhost:8081/repository/Build_Artifacts/05baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36.zip",
      "path" : "05baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36.zip",
      "id" : "T0lDX0J1aWxkX0FydGlmYWN0czphNDc1N2JjYWE2MmI2MzA2MDdlMTA1NGE4NTk1MDQ1OQ",
      "repository" : "Build_Artifacts",
      "format" : "raw",
      "checksum" : {
        "sha1" : "040e517528c05ca335a38e98c7ab8673773314bd",
        "sha512" : "652e624873da778dce4bc417cacfead12bdd2ad01e9f7f77cba80270caa0f99caa09d4fb53f584a4da9991186f78bd7b55da1e4ed3f22e026b8333dd332b3b83",
        "sha256" : "b66805d15702505f708b90f6e1169c79390d59b9f8cadc9ab1852f48eeabbfe2",
        "md5" : "60fdd406e026330c2cfbec3e15e05414"
      }
    } ]
  }, {
    "id" : "T0lDX0J1aWxkX0FydGlmYWN0czpkMDY0ODA0YThlZDVhZDZlNjhmZGU5MWNmM2NiZTgzMw",
    "repository" : "Build_Artifacts",
    "format" : "raw",
    "group" : "/",
    "name" : "15baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_2.zip",
    "version" : null,
    "assets" : [ {
      "downloadUrl" : "http://localhost:8081/repository/Build_Artifacts/15baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_2.zip",
      "path" : "15baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_2.zip",
      "id" : "T0lDX0J1aWxkX0FydGlmYWN0czpiZTc2NTljNDhiODY4YWQ0YjZiZjIyMDQyMmU5NWM2Ng",
      "repository" : "Build_Artifacts",
      "format" : "raw",
      "checksum" : {
        "sha1" : "040e517528c05ca335a38e98c7ab8673773314bd",
        "sha512" : "652e624873da778dce4bc417cacfead12bdd2ad01e9f7f77cba80270caa0f99caa09d4fb53f584a4da9991186f78bd7b55da1e4ed3f22e026b8333dd332b3b83",
        "sha256" : "b66805d15702505f708b90f6e1169c79390d59b9f8cadc9ab1852f48eeabbfe2",
        "md5" : "60fdd406e026330c2cfbec3e15e05414"
      }
    } ]
  }, {
    "id" : "T0lDX0J1aWxkX0FydGlmYWN0czplMDE4OGVkMDcyOGZhNjhmNDExNzU2OGU1MjQ2NjZiYg",
    "repository" : "Build_Artifacts",
    "format" : "raw",
    "group" : "/",
    "name" : "25baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_4.zip",
    "version" : null,
    "assets" : [ {
      "downloadUrl" : "http://localhost:8081/repository/Build_Artifacts/25baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_4.zip",
      "path" : "25baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_4.zip",
      "id" : "T0lDX0J1aWxkX0FydGlmYWN0czoyZmZmNTA5YTdjMmE5ZWJlYTRmNDQ1NjJiZmI5N2QxMA",
      "repository" : "Build_Artifacts",
      "format" : "raw",
      "checksum" : {
        "sha1" : "040e517528c05ca335a38e98c7ab8673773314bd",
        "sha512" : "652e624873da778dce4bc417cacfead12bdd2ad01e9f7f77cba80270caa0f99caa09d4fb53f584a4da9991186f78bd7b55da1e4ed3f22e026b8333dd332b3b83",
        "sha256" : "b66805d15702505f708b90f6e1169c79390d59b9f8cadc9ab1852f48eeabbfe2",
        "md5" : "60fdd406e026330c2cfbec3e15e05414"
      }
    } ]
  }, {
    "id" : "T0lDX0J1aWxkX0FydGlmYWN0czo2NTRiYjdkMGE1OTIxMzg1OWZhMTVkMzNmYWU1ZmY3OA",
    "repository" : "Build_Artifacts",
    "format" : "raw",
    "group" : "/",
    "name" : "35baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_3.zip",
    "version" : null,
    "assets" : [ {
      "downloadUrl" : "http://localhost:8081/repository/Build_Artifacts/35baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_3.zip",
      "path" : "35baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_3.zip",
      "id" : "T0lDX0J1aWxkX0FydGlmYWN0czoyNGMxZmUzMzVkMjY2NTJmZDRjOWZjZWZlYTQ4ODNhOA",
      "repository" : "Build_Artifacts",
      "format" : "raw",
      "checksum" : {
        "sha1" : "040e517528c05ca335a38e98c7ab8673773314bd",
        "sha512" : "652e624873da778dce4bc417cacfead12bdd2ad01e9f7f77cba80270caa0f99caa09d4fb53f584a4da9991186f78bd7b55da1e4ed3f22e026b8333dd332b3b83",
        "sha256" : "b66805d15702505f708b90f6e1169c79390d59b9f8cadc9ab1852f48eeabbfe2",
        "md5" : "60fdd406e026330c2cfbec3e15e05414"
      }
    } ]
  } ],
  "continuationToken" : null
}

//Output when using res.json()
{
  items: [
    {
      id: 'T0lDX0J1aWxkX0FydGlmYWN0czpmODk4YjM5MDNjYjk5YzU5NjgzNTk3ZWRjNWFmZjliYw',
      repository: 'Build_Artifacts',
      format: 'raw',
      group: '/',
      name: '05baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36.zip',
      version: null,
      assets: [Array]
    },
    {
      id: 'T0lDX0J1aWxkX0FydGlmYWN0czpkMDY0ODA0YThlZDVhZDZlNjhmZGU5MWNmM2NiZTgzMw',
      repository: 'Build_Artifacts',
      format: 'raw',
      group: '/',
      name: '15baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_2.zip',
      version: null,
      assets: [Array]
    },
    {
      id: 'T0lDX0J1aWxkX0FydGlmYWN0czplMDE4OGVkMDcyOGZhNjhmNDExNzU2OGU1MjQ2NjZiYg',
      repository: 'Build_Artifacts',
      format: 'raw',
      group: '/',
      name: '25baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_4.zip',
      version: null,
      assets: [Array]
    },
    {
      id: 'T0lDX0J1aWxkX0FydGlmYWN0czo2NTRiYjdkMGE1OTIxMzg1OWZhMTVkMzNmYWU1ZmY3OA',
      repository: 'Build_Artifacts',
      format: 'raw',
      group: '/',
      name: '35baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36_3.zip',
      version: null,
      assets: [Array]
    }
  ],
  continuationToken: null
}

As you can see from above res.text() provides verified/accurate JSON and res.json() does not. Without having accurate JSON makes parsing that much more difficult.

END UPDATE

I am anywhere close to getting this? Can anyone point me in the right direction or give me a simple example of creating a buffered object from a fetch request I can work with? Any and all help is appreciated.

if you need json why not just use `const data = await res.json()` instead of text? This way it will be object and you can use it. — Ashish Modi, Apr 15 '20 at 19:11
When I use res.json(), the object doesn't come back as legit JSON. See above where I added the output when using res.json() instead of rest.text() — Seth0080, Apr 15 '20 at 19:43
Dont see anything wrong with ouput from json. If you do `JSON.stringify(await res.json())` you will see a proper json. When you have nested json objects, console.log prints them as objects like [Array] — Ashish Modi, Apr 15 '20 at 19:55
ahh I see what you are saying...let me tool around with this to see if I can parse it correctly. I will comment back shortly...Thanks! — Seth0080, Apr 15 '20 at 20:11
So I guess where my newb shows here...when I have the same JSON set to a variable, i.e. "const data = JSON" I am able to parse it using the following notation: console.log(data.items[0].name) and my output shows "05baf31c20d15edb2c477fa4e7bd2427504d3dba_DATE_2020-04-10_TIME_11_09_36.zip" which is what I expect. However, when I try and do the same parsing using function return output I get "undefined"....I just don't understand why this is happening....? I guess I can close this ticket and start another one with that question.... — Seth0080, Apr 15 '20 at 20:41

score 0 · Answer 1 · answered Apr 16 '20 at 11:11

0

Per Ashish Modi.....

Dont see anything wrong with ouput from json. If you do JSON.stringify(await res.json()) you will see a proper json. When you have nested json objects, console.log prints them as objects like [Array]

answered Apr 16 '20 at 11:11

Seth0080

147
3
10

Creating buffer object from fetch response Javascript/Nodejs

1 Answers1