Puppeteer - How to use page.click() inside page.evaluate()

Question

I am scraping a table and each row has a button to show a modal with information. I need to scraping the information from the modal for each row but I dont know how to open the modal. I have tried with page.click(selector) inside page.evaluate() but It didnt work.

I have tested the next code.

const users = await page.evaluate(() => {
  const usersTable = Array.from(document.querySelectorAll('table#grilla > tbody > tr'))
  const userInfo = usersTable.map(async userTable => {
    await page.click('td > a#modificar-2')
    await page.click('div > button[type=submit].close')
    const username = userTable.children[2].innerText
    const firstName = userTable.children[4].innerText
    const lastName = userTable.children[5].innerText
    const email = userTable.children[6].innerText
    const estado = userTable.children[7].innerText
    const fullName = firstName + lastName
    return { username, fullName, email, estado }
  })
  return userInfo
})

I dont know how to pass page.click() or another option inside page.evaluate()

So 1. You won't be able to use page inside the evaluate, because the code inside the evaluate runs inside the browser. Can't you click those elements as if you were in the browser? — hardkoded, Jan 22 '20 at 19:42
Thanks for your response, I have analyzed it and I go to try use page.$$eval() to get all rows and after use a for() to extract specific information of the row and use page.click to access for the modal information. Let me know if i am wrong. — Arsenio Aguirre, Jan 22 '20 at 20:29

score 3 · Answer 1 · answered Jan 27 '20 at 22:11

If you use page.evaluate() you're switching the context from node puppeteer to browser, so you have to use JS native functions like click: document.querySelector(selector).click().

If you have errors like Error: Evaluation failed: TypeError: Cannot read property 'click' of null probably the element you wanted to click isn't on the page (or it's hidden or something).

score 1 · Answer 2 · answered Jan 23 '20 at 03:52

1

From inside that loop you would do:

userTable.querySelector('td > a#modificar-2').click()

(no await)

answered Jan 23 '20 at 03:52

pguardiario

53,827
19
119
159

Hi, thanks for your response. I have tried and it shows: ``Error: Evaluation failed: TypeError: Cannot read property 'click' of null`` – Arsenio Aguirre Jan 23 '20 at 14:24
That means the element isn't there. – pguardiario Jan 24 '20 at 00:55

score 0 · Answer 3 · answered Nov 11 '20 at 07:15

You can return css selectors of elements you want to click from page.evaluate and then perform page.click on them. For getting css selector for the element you can use the code from an answer to this question:

const elemToClick = page.evaluate(() => {
  let elem;

  //...

  return cssPath(elem);

  function cssPath(el) {
    if (!(el instanceof Element)) return;
    var path = [];
    while (el.nodeType === Node.ELEMENT_NODE) {
      var selector = el.nodeName.toLowerCase();
      if (el.id) {
        selector += '#' + el.id;
        path.unshift(selector);
        break;
      } else {
        var sib = el,
          nth = 1;
        while ((sib = sib.previousElementSibling)) {
          if (sib.nodeName.toLowerCase() == selector) nth++;
        }
        if (nth != 1) selector += ':nth-of-type(' + nth + ')';
      }
      path.unshift(selector);
      el = el.parentNode;
    }
    return path.join(' > ');
  }
});

page.click(elemToClick);

Puppeteer - How to use page.click() inside page.evaluate()

3 Answers3