find some value in javascript, in the response form

Question

I have an url www.example.com/test so by using robobrowsker to visit this url, I find some js in response and it contains something like this

var token = _.unescape("<input name="__RequestVerificationToken" type="hidden" value="wi5U8xXijdXRrPR4aG84OAjSLsuS1YqTV4X7VLDnWeuwr72D39H-KXBsyG7eZEZPT7YXW7GF26IiQBrW0vcEZd5Bqrjof_CVEUFRTDPS4rx68Opmi6juZXnGDEtb9nsBXxM4Why2WNlflqFM6purXw2" />");
    aw.antiforgeryToken[$(token).attr('name')] = $(token).val();

I want to get 'wi5U8xXijdXRrPR4aG84OAjSLsuS1YqTV4X7VLDnWeuwr72D39H-KXBsyG7eZEZPT7YXW7GF26IiQBrW0vcEZd5Bqrjof_CVEUFRTDPS4rx68Opmi6juZXnGDEtb9nsBXxM4Why2WNlflqFM6purXw2'

I tried this

browser=RoboBrowser()
browser.open('https://www.example.com/test')
result=browser.find('script',{'name':'__RequestVerificationToken'})

This gives 'None'

so how can I do this ?

thanks

Answer 1

br.find works on html, and as the stuff you want is inside a JS call so we can't use it.

so other options are

1. use rejex (wiz. a bit hardcoded in my opinion)

   By finding the parent node in which the node which eventually contains the data you want is present, and then find that string i.e. 'wi5U8xXijdXRrPR4aG84OAjSLsuS1YqTV4X7VLDnWeuwr72D39H-KXBsyG7eZEZPT7YXW7GF26IiQBrW0vcEZd5Bqrjof_CVEUFRTDPS4rx68Opmi6juZXnGDEtb9nsBXxM4Why2WNlflqFM6purXw2' via regex

2. lxml.html (xpath)
   it is other way which I MAY prefer is lxml.html or import html from lxml one and the same thing

here is a bit of representation of it.

data = lmxl.html(parsedData)
stuff = data.xpath('XPATH to you data')

you can find more here Can I parse xpath using python, selenium and lxml? and have a look in docs as well

I hope I was helpful.

cheers.