How do you send a POST request in Puppeteer?

Question

I use this code for sending a GET request:

(async() => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('https://www.example.com/search');
    const data = await page.content();
    browser.close();
    res.send(data);
})();

How do I send a POST request?

Answer 1

Getting the "order" right can be a bit of a challenge. Documentation doesn't have that many examples... there are some juicy items in the repository in the example folder that you should definitely take a look at.

https://github.com/GoogleChrome/puppeteer/tree/main/examples

Here is the example; place the following into an async block:

// Create browser instance, and give it a first tab
const browser = await puppeteer.launch();
const page = await browser.newPage();

// Allows you to intercept a request; must appear before
// your first page.goto()
await page.setRequestInterception(true);

// Request intercept handler... will be triggered with 
// each page.goto() statement
page.on('request', interceptedRequest => {

    // Here, is where you change the request method and 
    // add your post data
    var data = {
        'method': 'POST',
        'postData': 'paramFoo=valueBar&paramThis=valueThat'
    };

    // Request modified... finish sending! 
    interceptedRequest.continue(data);
});

// Navigate, trigger the intercept, and resolve the response
const response = await page.goto('https://www.example.com/search');     
const responseBody = await response.text();
console.log(responseBody);

// Close the browser - done! 
await browser.close();

Answer 2

There's a quirk with the way setRequestInterception and the 'request' event work. Once activated, Puppeteer will send the POST data to every resource on the page, not just the original requested page. I was having the issue that all of my page resources (scripts, CSS) were failing to load once I added POST data in Puppeteer.

Since I only want to apply POST data to the first request, this code worked for me:

// Used for serializing POST parameters from an object
const querystring = require('querystring');

// ...

const browser = await puppeteer.launch();
const page = await browser.newPage();

let postData = {a: 1, b: 2};

await page.setRequestInterception(true);

page.once('request', request => {
    var data = {
        'method': 'POST',
        'postData': querystring.stringify(postData),
        'headers': {
            ...request.headers(),
            'Content-Type': 'application/x-www-form-urlencoded'
        },
    };

    request.continue(data);

    // Immediately disable setRequestInterception, or all other requests will hang
    page.setRequestInterception(false);
});

const response = await page.goto('https://www.example.com/');

Answer 3

Here the complete example with Puppeteer 2.0.0 :

    const puppeteer = require("puppeteer");
    const devices = require("puppeteer/DeviceDescriptors");

    async function main() {
      const browser = await puppeteer.launch({
        args: ["--enable-features=NetworkService", "--no-sandbox"],
        ignoreHTTPSErrors: true
      });
      const page = await browser.newPage();

      await page.setRequestInterception(true);

      page.once("request", interceptedRequest => {
        interceptedRequest.continue({
          method: "POST",
          postData: "foo=FOO&bar=BAR",
          headers: {
            ...interceptedRequest.headers(),
            "Content-Type": "application/x-www-form-urlencoded"
          }
        });
      });

      const response = await page.goto("https://postman-echo.com/post");

      console.log({
        url: response.url(),
        statusCode: response.status(),
        body: await response.text()
      });

      await browser.close();
    }

    main();

Note that if you check response.request().method() it won't be updated (still GET)

Answer 4

You can use the Page.evaluate() method to run Fetch API inside the browser. For example:

const postResponse = await page.evaluate( async (param1, param2, param3) => {
  const response = await fetch("https://www.example.com/add", {
    "headers": {
      "content-type": "application/x-www-form-urlencoded; charset=UTF-8"
    },
    "body": `param1=${param1}&param2=${param2}&param3=${param3}`,
    "method": "POST"
  });
  const data = await response.json();
  return data;
}, param1, param2, param3);