How to get Puppeteer waitForNavigation Working after click

Question

I am trying to get puppeteer to wait for the navigation to finish before moving on to the next statement. Based on the Docs for waitForNavigation() , the code should work below. but it just skips to the next statement and I have to use a workaround to wait for a specific URL in the response.

I have tried all the waituntil options as well
( load, domcontentloaded, networkidle0 and networkidle2 ) .

Any ideas how I could get that working properly is appreciated.

const browser = await puppeteer.launch({
  headless: false,
})
const page = await browser.newPage()
const home = page.waitForNavigation()
await page.goto(loginUrl)
await home

const login = page.waitForNavigation()
await page.type('#email', config.get('login'))
await page.type('#password', config.get('password'))
await page.click('#submitButton')
await login // << skips over this

// the following line is my workaround and it works , but ideally I don't want 
// to specify the expected "after" page each time I navigate
await page.waitForResponse(request => request.url() === 'http://example.com/expectedurl')

Possible duplicate of [How to login in Puppeteer?](https://stackoverflow.com/questions/50074799/how-to-login-in-puppeteer) — Md. Abu Taher, Sep 06 '18 at 20:57
It could be there are no navigation at all. Maybe an example real URL or more details about the target website like framework, hash url etc might be helpful. — Md. Abu Taher, Sep 07 '18 at 04:55

Grant Miller · Accepted Answer · 2019-07-10T22:11:20.147

19

The function page.waitForNavigation() waits for navigation to begin and end.

The navigation has already been initiated with page.click().

Therefore, you can use Promise.all() to avoid the race condition between the mentioned functions:

const browser = await puppeteer.launch({
  headless: false,
});

const page = await browser.newPage();

await page.goto(loginUrl);

await page.type('#email', config.get('login'));
await page.type('#password', config.get('password'));

await Promise.all([
  page.click('#submitButton'),
  page.waitForNavigation({
    waitUntil: 'networkidle0',
  }),
]);

await browser.close();

edited Jul 10 '19 at 22:11

answered Sep 06 '18 at 21:29

Grant Miller

27,532
16
147
165

Thanks Grant , interestingly is doesn't work! I don't know why , but it skips over. My guess is that after the logon it is finishing and then going to another page. – Martin Thompson Sep 06 '18 at 22:01
@MartinThompson That sounds like a possibility. You can check for redirects with [`request.redirectChain()`](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#requestredirectchain). – Grant Miller Sep 07 '18 at 01:48
@GrantMiller I have some doubts regarding that, can we chat? It would be really helpful for me. – Suraj Jain Feb 27 '19 at 20:55
would be great to also show how to get the click result – CodingYourLife Feb 26 '21 at 03:14
@CodingYourLife check my answer – Mouneer May 11 '21 at 14:31

score 1 · Answer 2 · answered May 29 '21 at 18:56

I have been going through the same problem, I used pending-xhr-request

it solved many problems when the requests were expected, but when I have late requests I have faced many problems, it took me a while to solve the problem so I built a package Puppeteer-response-waiter to do that

const puppeteer = require('puppeteer');
const {ResponseWaiter} = require('puppeteer-response-waiter');

let browser = await puppeteer.launch({ headless: false });
let page = await browser.newPage();
let responseWaiter = new ResponseWaiter(page);
await page.goto('http://somesampleurl.com');
// start listening
responseWaiter.listen();
// do something here to trigger requests
await responseWaiter.wait();
// all requests are finished and responses are all returned back

// remove listeners
responseWaiter.stopListening();
await browser.close();

hope this will solve your problem.

How to get Puppeteer waitForNavigation Working after click

2 Answers2

Linked