0

What I am trying to do is to get a request from a server using fetch in Javascript. The problem is that the answer is coming like a partial-response with xml, so the answer don't come at once, but fetch only give me the first part.

I already tried things like asynchronous things (like await, or then), but I always have only the first part of the answer.

Here is my code :

let response = await fetch(url, {
        method: 'POST',
        body: // my body
        headers: {
            'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
            'Connection': 'close',
            'Faces-Request': 'partial/ajax',
            'X-Requested-With': 'XMLHttpRequest',
            'Accept': 'application/xml, text/xml, */*; q=0.01',
            'Accept': '*/*',
            'Accept-Encoding': 'gzip, deflate',
            // then cookie, user-agent ...
        }});

I also tried using XMLHttpRequest, but I need to use a Cookie header, that I can't use with it.

For the answer, I am expecting something like this (I replaced the personal informations with _) :

<?xml version='1.0' encoding='UTF-8'?>
<partial-response id="j_id1">
    <changes>
        <update id="form:messages">
            <![CDATA[_]]></update>
        <update id="form:j_idt117">
            <![CDATA[_]]></update>
        <update id="form:j_idt133">
            <![CDATA[_]]></update>
        <update id="form:j_idt228">
            <![CDATA[_]]></update>
        <update id="j_id1:javax.faces.ViewState:0"><![CDATA[_]]></update>
    </changes>
</partial-response>

But I get :

<?xml version='1.0' encoding='UTF-8'?>
<partial-response id="j_id1">
    <changes>
        <update id="form:j_idt228">
            <![CDATA[_]]></update>
        <update id="j_id1:javax.faces.ViewState:0"><![CDATA[_]]></update>
    </changes>
</partial-response>

So the question is, does fetch have something to help me with it ? I thought first of a wait before beginning to hear for the answer, but I find no documentation on that kind of thing.

Thank you in advance for your help !

Nicolas B.
  • 1
  • 3
  • You need to check this in your backend. What happen when you send the request with postman or other similar tools? – CodeThing Mar 07 '23 at 17:51
  • I don't think Fetch can set the cookie header, either, for the same security reason as XHR. If you want to set a cookie you have to assign `document.cookie`. Then use the `credentials:` option so that Fetch will send cookies. – Barmar Mar 07 '23 at 17:55
  • Using burp, I get the XML part that is shown in the XML which is expecting. – Nicolas B. Mar 07 '23 at 17:56
  • I tried ading "credentials: 'include'", but I still have the same answer. But I don't think that the problem come from the cookie, as in this case, the cookie is used for some kind of authentification, but it is still giving me the good status code with an answer, just not all the answer. – Nicolas B. Mar 07 '23 at 18:01

1 Answers1

0

Can you try this code - node-fetch:

const fetch = require('node-fetch');

async function fetchChunked(url, options) {
  const response = await fetch(url, options);
  if (!response.ok) {
    throw new Error(`Failed to fetch ${url}: ${response.status} ${response.statusText}`);
  }

  const chunks = [];
  let length = 0;
  for await (const chunk of response.body) {
    chunks.push(chunk);
    length += chunk.length;
  }

  const buffer = Buffer.concat(chunks, length);
  const data = buffer.toString('utf-8');
  return data;
}

const url = 'https://example.com';
const options = {
  method: 'POST',
  body: // my body
  headers: {
    'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
    'Connection': 'close',
    'Faces-Request': 'partial/ajax',
    'X-Requested-With': 'XMLHttpRequest',
    'Accept': 'application/xml, text/xml, */*; q=0.01',
    'Accept': '*/*',
    'Accept-Encoding': 'gzip, deflate',
    // then cookie, user-agent ...
  },
};

const data = await fetchChunked(url, options);
console.log(data);
Greg
  • 1,401
  • 11
  • 11
  • When doing this, I have an error "TypeError: response.body.getReader is not a function", am I doing something wrong ? – Nicolas B. Mar 07 '23 at 18:04
  • I updated the code with: let responseClone = response.clone(); let reader = responseClone.body.getReader(); Can you try, please? If this is code from node.js, not the browser it may require some changes. Are you using node-fetch? – Greg Mar 07 '23 at 18:35
  • Yes I am using fetch from le node-fetch library. I tried using the updated code, but still the same, "responseClone.body.getReader is not a function". – Nicolas B. Mar 08 '23 at 08:53
  • Trying to figure out if simple await response.text() will do the trick, I expect your incorrect result is coming through this, right? I update the code in the answer. – Greg Mar 08 '23 at 18:28
  • I tried this, as in your code update, I obtain as result "class java.lang.IndexOutOfBoundsException<![CDATA[Index: 0, Size: 0]]>" I tried to your update with axios and get an error "response.data.on is not a function", don't really know why – Nicolas B. Mar 09 '23 at 13:49
  • Let's keep on trying, I updated my answer, try this – Greg Mar 16 '23 at 20:47
  • Thank you for your answer, it is running, but I still have the same response as in my initial question, which is incomplete. I think I am missing something. I tried too using the same process but using python and I also have the same incomplete answer, but if I put the same request in Burp, I get the complete answer. – Nicolas B. Mar 18 '23 at 09:39