How to click a 'submit' button in page with node scraper and then retrieve the page it goes to

Question

I am building a scraper in node, but I have gotten stumped. I go to this address: https://ariisp1.oklahomacounty.org/AssessorWP5/DefaultSearch.asp and I want to simulate putting an address in the second textbox and clicking the subsequent 'Submit' button. I can successfully find the textbox and the following button, but I cannot figure out how to simulate the 'click' function and retrieve the url it goes to. There is no href associated with the button because the url it goes to is determined by the address that is put in. Any ideas?

var request = require('request');
var cheerio = require('cheerio');


request.post('https://ariisp1.oklahomacounty.org/AssessorWP5/DefaultSearch.asp', /*{
  form: {
    FormattedLocation: '2333 Nw 32 St'
    //btnSubmit: 'Submit'
  }
}, */
function (err, res, body) {
    let $ = cheerio.load(body);


    $("input[name=FormattedLocation]").text('2331 nw 32 st');
    var x = $("input[name=FormattedLocation]").text();
    var y = $("input[name=FormattedLocation]").next().attr('type');

    console.log(y);//successfully gets the 'Submit' button

    //code to click button and get the page it goes to goes here
})

i played around a while back with a headless browser that used cheerio, called [zombie](https://github.com/assaf/zombie). Doubt you'll be able to get cheerio alone to do what you're trying to do...it's a fairly lightweight DOM parser, not really meant for submitting forms and such. — David784, Jul 11 '18 at 22:23
You would need to figure out what clicking the button does (GET/POST) and make that request with `request` — pguardiario, Jul 11 '18 at 23:15

score 1 · Accepted Answer · answered Jul 12 '18 at 01:58

You don't need to click the button with cheerio, what you need to do is make the POST request and then parse the results. It's not fully clear what you're trying to do with the results, and your sample code is already doing basically that. The difficult part is that the form response you get is a total mess of HTML and you've not told us what you are trying to extract from it.

request.post('https://ariisp1.oklahomacounty.org/AssessorWP5/AddressSearch.asp', { form: { FormattedLocation: '2333 Nw 32 St' }}, (err, res, body) => console.log(body));

If you run that, you'll see that your request happens and gets the HTML coming back as if you'd clicked the Submit button in question. From there, what you do with cheerio is up to what you want to extract from the response HTML.

yeah i had to get the url from the form's post action location for the url and add that in with the address. totally forgot about post action. — Rainhider, Jul 19 '18 at 17:18

How to click a 'submit' button in page with node scraper and then retrieve the page it goes to

1 Answers1