1

I want to extract the text result with the "input#Readonly1" selector. But unfortunately I get an empty text. (value contain an empty text)

const puppeteer = require('puppeteer');
var sleep = require('system-sleep');

(async () => {
    const browser = await puppeteer.launch({ headless: false });
    const page = await browser.newPage();
    await page.goto('https://www.mobilit.fgov.be/WebdivPub_FR/wmvpstv1_fr?SUBSESSIONID=16382865', { timeout: 8000 });

    const elementHandle = await page.$(
    'frame[name="AppWindow"]',);
  const frame = await elementHandle.contentFrame();
  await frame.type("input[name='Writable3']", 'VF33C9HYB84113557', { delay: 5 });
  await frame.$eval("button[name='btnOphalen']", el => el.click());
  const element = await frame.waitForSelector("input#Readonly1", {visible: true}); // select the element
  const value = await element.evaluate(el => el.textContent);
  console.log(value);
  
  await browser.close();
})();

I think this line of code is incorrect:

const element = await frame.waitForSelector("input#Readonly1", {visible: true});
Mohamed abdelmoula
  • 107
  • 6

2 Answers2

3

There seem to be two errors:

  1. The following line (as you mention in your question):

    const element = await frame.waitForSelector("input#Readonly1", {visible: true});

    Checking the page you're using, it seems that input#Readonly1 is always visible, it only changes the value when you perform the query.

    Updated taking into account @ggorlen answer and comments

    You can use waitForFunction:

    await frame.click("button[name='btnOphalen']");
await frame.waitForFunction(
  'document.querySelector("input#Readonly1").value.trim().length > 0',
  { polling: 'mutation' }
);
const element = await frame.$("input#Readonly1");
    Why { polling: 'mutation' }?

    According to FrameWaitForFunctionOptions:

    • raf - to constantly execute pageFunction in requestAnimationFrame callback. This is the tightest polling mode which is suitable to observe styling changes.
    • mutation - to execute pageFunction on every DOM mutation.

  2. According to mdn: Node.textContent

    For other node types, textContent returns the concatenation of the textContent of every child node, excluding comments and processing instructions. (This is an empty string if the node has no children.)

    Since you are retrieving the text inside an input, use value instead of textContent

    const value = await element.evaluate(el => el.value);
Brian
  • 309
  • 2
  • 10
  • I tried with value instead of textContent but it didn't work – Mohamed abdelmoula Oct 01 '22 at 17:15
  • 1
    @Mohamedabdelmoula When viewing the page, you were right in the line that you indicate in your question as the wrong one, I update the answer there – Brian Oct 01 '22 at 17:19
  • 1
    thank you very much but it still doesn't work – Mohamed abdelmoula Oct 01 '22 at 17:31
  • @Mohamedabdelmoula I left the correct answer there, I got a bit confused with the puppeteer api, let me know if it still doesn't work – Brian Oct 01 '22 at 17:39
  • I added `await delay(1000);` after `await frame.$eval("button[name='btnOphalen']", el => el.click());` and it works. But I'm wondering if there isn't another way to not use delay ? – Mohamed abdelmoula Oct 01 '22 at 18:16
  • I tried `const element = await frame.waitForFunction('document.querySelector("input#Readonly1").innerText.includes(":")');` but it doesn't work – Mohamed abdelmoula Oct 01 '22 at 18:19
  • 1
    Since that page changes values using the api, the events (change, input, ...) are not emitted, so the only thing I can think of is to use a mutation observer, but I'm not really sure if it works. – Brian Oct 01 '22 at 18:48
  • 1
    Unless you don't want to use the delay function because of some design decision in your code, I think it's the cleanest solution. – Brian Oct 01 '22 at 18:50
  • 1
    Sleeping is not the cleanest solution, it was [removed from the Puppeteer API](https://stackoverflow.com/a/73676564/6243352) for a reason. It is imprecise and either slows your script down for no reason or creates a possibility of failing when an unexpected delay arises. `waitForFunction` is better if you can find a predicate. I'll take a look here. Beyond that, though, the answer is correct about the `.value` issue and that there's no need to `waitForSelector` so it's helpful. – ggorlen Oct 01 '22 at 19:08
  • 1
    @Mohamedabdelmoula your `waitForFunction` can use `.value.includes()` and it should work, but you've probably already seen my solution. The issue with `:` is that maybe the format changes and `:` is no longer in the output, so I'd prefer a more general predicate like "wait for the value to change" or "wait until the contents are non-whitespace". – ggorlen Oct 01 '22 at 20:06
  • @ggorlen The sleep deprecation makes sense, checking around a bit, I was thinking of using `waitForResponse`, but using `waitForFunction` seems easier – Brian Oct 01 '22 at 21:05
  • 1
    `waitForResponse` is a good idea too, but the issue is that the response isn't a traditional JSON API response, so it seems easier to stick to working with the DOM. – ggorlen Oct 01 '22 at 21:07
  • 1
    @Mohamedabdelmoula and ggorlen thanks for the comments, there I updated the answer, tell me if there is something to correct – Brian Oct 01 '22 at 21:49
  • 2
    Great update, thanks for not sleeping. Extremely minor point, but if a future version of the site does have some default non-space value in the input, the `waitForFunction` predicate here will fail, while my approach of "wait for _any_ change whatsoever" should still work. But it's highly unlikely--just a minor nitpick. – ggorlen Oct 01 '22 at 23:50
1

The existing answer is correct that you would use .value rather than .textContent for getting the value from an input field. However, it also suggests a sleep/delay workaround in place of waiting for a proper predicate. Sleeping is inaccurate, resulting in either wasted time or false positives.

One better approach is using waitForFunction to compare the input's value before the click against its current value in a tight requestAnimationFrame loop. At the precise render that the value changes, the waitForFunction predicate will resolve and your code will continue on to the next line, taking no more or less time than it has to.

const puppeteer = require("puppeteer"); // ^18.0.4

let browser;
(async () => {
  browser = await puppeteer.launch({headless: true});
  const [page] = await browser.pages();
  const url = "https://www.mobilit.fgov.be/WebdivPub_FR/wmvpstv1_fr?SUBSESSIONID=16382865";
  await page.goto(url, {waitUntil: "load"});
  const el = await page.$('frame[name="AppWindow"]');
  const frame = await el.contentFrame();
  await frame.type("#Writable3", "VF33C9HYB84113557");
  const output = await frame.$("#Readonly1");
  const preValue = await output.evaluate(el => el.value);
  await frame.click("#btnOphalen");
  await frame.waitForFunction(preValue =>
    document.querySelector("#Readonly1").value !== preValue,
    {}, preValue
  );
  const value = await output.evaluate(el => el.value);
  console.log(value); // => Marque et type : PEUGEOT 307
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close())
;

To get all fields, try:

await frame.$$eval('[id^="Readonly"]', els => els.map(el => el.value));
ggorlen
  • 44,755
  • 7
  • 76
  • 106