What are the max chars in a URL string that can be passed to the Puppeteer page.goto(url) function

Question

Background

We are using Puppeteer to render PDFs on a Node server. We are using an API to pass large query strings to the API which is passed to Puppeteer. Once Puppeteer renders the web page, the data in the GET query string is pulled into the HTML page rendered so the data in the page is populated dynamically. Once the page renders, Puppeteer converts it to a PDF and it is downloaded to the client.

Problem

We realized that when the requests are very large it breaks the browser when we hit the API with a GET request. To overcome this we are hitting the API as a POST and hashing the data so it can be rendered later.

This got us wondering if there is a max char for the puppeteer function rendering the web page used to render a PDF.

Example Code

const browser = await puppeteer.launch({
          args: ['--no-sandbox', '--disable-setuid-sandbox'],
          ignoreHTTPSErrors: true,
          dumpio: false
        });

        const page = await browser.newPage();

        const data = reqMethod === 'POST' ? req.body : JSON.parse(req.query.data);

        const {pdfOptions, ...templateData} = data;

        const url = `${PDF_API_PROD}/${template}?data=${JSON.stringify(templateData)}`;

        await page.goto(url);

        const pdfBuffer = await page.pdf({
          format: 'A4',
          margin: {
            top: '20px',
            left: '20px',
            right: '20px',
            bottom: '20px',
          },
          ...pdfOptions,
        });

Question

After looking at the code above you will see that we are passing the data object directly into the URL as a GET param. This will be used to render the web page with Puppeteer.

Once the web page is rendered with Puppeteer the data in the GET string will be pulled into the web page with JavaScript in order to render the page dynamically.

What is the max chars that can be passed into the Puppeteer function await page.goto(url);?

Answer 1

There is no hard limit built into the browser. I was able to send URLs of a length of up to 2000000 characters to a server myself without any problems. Even after that, I only had trouble because it just takes some time to send the data.

If you are having trouble sending large ULRs, it is most likely one of the following two things:

1. The server is not properly configure to receive the amount of data.

To receive that much data, you have to properly configure your server. By default, most server will cap the data which can be send via the URL.

2. You are hitting a timeout

Keep in mind, that sending a few MB of data, might take some time depending on your internet connection and the server upload speed. It might also be slower to send the data in the head of the HTTP request instead of sending it as a stream inside the body. In my test cases, this was the limiting factor.

Therefore: Most likely, the problem you are encountering is not related to puppeteer but to the receiving end.

What puppeteer does

As you are thinking that puppeteer might truncate the URL: This is not the case. puppeteer is just a wrapper around the DevTools Protocol. Puppeteer will take the URL argument, wrap it as part of the payload via JSON.stringify and send it to the browser. I doubt that the DevTools Protocol has any limitations built into Page.navigate. Therefore, there should be no "library-specific" introduced though puppeteer here.