0

I am trying to scrape a dynamic site using puppeteer in node, but not able to click the required elements no matter what. Please help!

// this script parses the data from https://excise.wb.gov.in/CHMS/Public/Page/CHMS_Public_Hospital_Bed_Availability.aspx
const puppeteer = require('puppeteer');

(async function scrape() {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();
  // page.setDefaultNavigationTimeout(90000);
  const url = "https://excise.wb.gov.in/CHMS/Public/Page/CHMS_Public_Hospital_Bed_Availability.aspx";
  await page.goto(url, {waitUntil: 'networkidle2', timeout: 0});
  // await page.waitForNavigation({ waitUntil: 'networkidle2' })
  await page.waitForSelector('#ctl00_ContentPlaceHolder1_ddl_District');
  await page.click('#ctl00_ContentPlaceHolder1_ddl_District');
})();

Tried this too, but no luck:

  const selectButt = await page.evaluateHandle(() =>
    document.querySelector('#ctl00_ContentPlaceHolder1_ddl_District')
  );
  await selectButt.click();
Vipin Verma
  • 5,330
  • 11
  • 50
  • 92

1 Answers1

2

It's a <select>, you probably don't want to click on it, but choose an option instead:

await page.select('#ctl00_ContentPlaceHolder1_ddl_District', '020'); // ALIPURDUAR

Docs for page.select

Triggers a change and input event once all the provided options have been selected.

blex
  • 24,941
  • 5
  • 39
  • 72
  • I tried it, did not work. lemme increase timeout – Vipin Verma May 16 '21 at 23:03
  • Why is `await page.click('#ctl00_ContentPlaceHolder1_rdo_Govt_Flag_2');` not working. I checked there is no special function for selecting radio button. – Vipin Verma May 16 '21 at 23:07
  • ` await page.evaluate(() => { var test = document.querySelector('#ctl00_ContentPlaceHolder1_rdo_Govt_Flag_2') test.click(); })` worked for the radio button – Vipin Verma May 16 '21 at 23:11
  • 1
    I'm guessing it's because Puppeteer can't access it because of the CSS (it's positioned absolutely with a z-index of -1). But clicking on the label seems to work: `await page.click('[for=ctl00_ContentPlaceHolder1_rdo_Govt_Flag_2]');` – blex May 16 '21 at 23:17
  • Can i request you to help with this too? https://stackoverflow.com/questions/67562491/puppeteer-page-evaluate-not-working-after-waitfornavigation-debug-not-working – Vipin Verma May 16 '21 at 23:52