How to log the json response body in playwright after a button click?

Question

I am trying to log the response body from an api call that is initiated by a button click in playwright. I have been at this for a while, with no results. I can log the response status, request headers and response headers, But not the response. Although, running a headed browser I can see the JSON response in Network tab inspect window.

await Promise.all([
   page.waitForResponse(resp => resp.url().includes('https://example.com/user-check') && 
   resp.status() === 200 && resp.request().method() === 'POST')    
   .then(async resp => {
     console.log("Response URL:", resp.url()); // this works and logs the URL of the response
     console.log("Response status:", resp.status()); // this works and logs 200
     console.log("Response body:", resp.body()); // this fails 
   }),
   page.click('#clickButton'),
 ]);

I tried resp.body(), resp.json(), resp.text() all failed with the same error below.

node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

response.json: Protocol error (Network.getResponseBody): No resource with given identifier found

I hope someone out there can help.

UPDATE: Based on the response headers, the content is gzip encoded. Therefore, I incorporated the solution provided by ggorlen as below.

  const responsePromise = page.waitForResponse(resp =>
      resp.url().includes("https://example.com/user-check") &&
      resp.status() === 200 &&
      resp.request().method() === "POST"
  );
  await page.click("#buttonClick");
  const resp = await responsePromise;
  console.log("Response URL:", resp.url());
  console.log("Response status:", resp.status());
  console.log("Response body:", zlib.gunzipSync(resp.body()));

I am guessing there is a specific way to decode the response body in playwright, because I got this error:

Response status: 200
TypeError [ERR_INVALID_ARG_TYPE]: The "buffer" argument must be of type string or an instance of Buffer, TypedArray, DataView, or ArrayBuffer. Received an instance of Promise

ggorlen · Answer 1 · 2023-03-19T16:04:31.397

It's difficult to help without the site, since there could be some additional behavior making the situation more complex than you assume, but the code should probably be arranged like:

const responsePromise = page.waitForResponse(resp =>
  resp.url().includes("https://example.com/user-check") &&
  resp.status() === 200 &&
  resp.request().method() === "POST"
);
await page.click("#clickButton");
const resp = await responsePromise;
console.log("Response URL:", resp.url());
console.log("Response status:", resp.status());
console.log("Response body:", resp.body());

Or, if you prefer Promise.all():

const [resp] = await Promise.all([
  page.waitForResponse(resp =>
    resp.url().includes("https://example.com/user-check") &&
    resp.status() === 200 &&
    resp.request().method() === "POST"
  ),
  page.click("#clickButton")
]);
console.log("Response URL:", resp.url());
console.log("Response status:", resp.status());
console.log("Response body:", resp.body());

The rule of thumb is never to mix await and then. Also, remove the async keyword on functions that don't use await.

Here's a minimal, complete example:

const playwright = require("playwright"); // ^1.30.1

const html = `<!DOCTYPE html><html><body>
<button id="clickButton">Go</button>
<script>
document.querySelector("button").addEventListener("click", e => {
  fetch("https://httpbin.org/post", {
    method: "POST",
    headers: {
      "Accept": "application/json",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({foo: 42})
  })
    .then(res => res.json())
    .then(d => console.log(d));
});
</script>
</body></html>`;

let browser;
(async () => {
  browser = await playwright.chromium.launch();
  const page = await browser.newPage();
  await page.setContent(html);
  const responsePromise = page.waitForResponse(resp =>
    resp.url().includes("https://httpbin.org/post") &&
    resp.status() === 200 &&
    resp.request().method() === "POST"
  );
  await page.click("#clickButton");
  const resp = await responsePromise;
  console.log("Response URL:", resp.url());
  console.log("Response status:", resp.status());
  console.log("Response body:", JSON.parse(await resp.body()));
  console.log("Request body:", JSON.parse(await resp.request().postData()));
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());

Output:

Response URL: https://httpbin.org/post
Response status: 200
Response body: {
  args: {},
  data: '{"foo":42}',
  files: {},
  form: {},
  headers: {
    Accept: 'application/json',
    'Accept-Encoding': 'gzip, deflate, br',
    'Content-Length': '10',
    'Content-Type': 'application/json',
    Host: 'httpbin.org',
    Origin: 'null',
    'Sec-Ch-Ua': '',
    'Sec-Ch-Ua-Mobile': '?0',
    'Sec-Ch-Ua-Platform': '',
    'Sec-Fetch-Dest': 'empty',
    'Sec-Fetch-Mode': 'cors',
    'Sec-Fetch-Site': 'cross-site',
    'User-Agent': '...',
    'X-Amzn-Trace-Id': '...'
  },
  json: { foo: 42 },
  origin: '98.51.1.252',
  url: 'https://httpbin.org/post'
}
Request body: { foo: 42 }

If this doesn't work when adapted to your use case, please provide enough information to reproduce the problem.

This comment in Playwright issue #3617 suggests that a navigation can cause this, so make sure you're properly handling such an event.

Thank you, and you were right, it is more complex. I found that the content is gzip encoded. Is there way to decode the the response before logging it? Here is a chunk of the response headers. Response headers: `{ expires: '0', date: 'Sun, 19 Mar 2023 19:00:39 GMT', 'strict-transport-security': 'max-age=31536000; includeSubDomains', 'x-content-type-options': 'nosniff', 'content-encoding': 'gzip', 'x-cdn': 'Imperva, Imperva', 'transfer-encoding': 'chunked', connection: 'keep-alive', }` — Adam, Mar 19 '23 at 19:12
I don't know much about that header, but my understanding is that's standard and handled by the request library. Are you expecting a JSON response or some other format? I can't help much further than this without seeing the site, a clear description of what you're trying to accomplish, an error message (if there is one and it's different than the one before) and a [mcve] of the failing code that I can run and experiment with. Thanks. — ggorlen, Mar 19 '23 at 19:15
[`'transfer-encoding': 'chunked'`](https://en.wikipedia.org/wiki/Chunked_transfer_encoding) seems relevant... are you expecting a stream of data? — ggorlen, Mar 19 '23 at 19:20
Yes, I am expecting a json response. I also think gzip decoding is handled by the library. The json response in this structure `{ "user": "external", "status": [ { "@Code": "available", "description": [ "" ], "effect": "[]", "permissions": {} } ], "id": "35901033", "status": "available", "phone": "1234567890", "support": "Unknown", "portal": "remote" }` — Adam, Mar 19 '23 at 19:48
Thanks, but I'm still missing the critical reproducibility and error diagnostic information necessary to help further. If you use my code and plug in your URL, you should see that response. If you don't, please share the URL, the exact error you're seeing and the steps I can take (i.e. runnable code) to reproduce that error on my machine so I can offer a solution. — ggorlen, Mar 19 '23 at 19:53

How to log the json response body in playwright after a button click?

1 Answers1