1

i've read many posts about this, but so far i couldn't find a solution. maybe i am too dumb, in that case, please point it out while beeing nice to me :D

this is my first question i post here, so be nice anyways :D

my problem is, i use selenium to open some websites and click some buttons. after that, i get the page source and parse it with jsoup for some content i need. this works, but there are pages, where i dont get the current dom, but the page source as it was sent first by the server. i can click on the stuff which was added by the javascripts, so selenium knows they are there. but i can't get the whole current html code. so basically, my question is:

how can i get with selenium what is shown when i right click on a page and click view page source? because what i see there differs from what i get when i execute getpagesource() method.

people answered some of the questions with things like: i have to wait, i have to execute the javascript by myself etc... but this doesn't help me to be honest. the button is there already (because the javascript already added it), i can navigate with selenium and click on it. when i load pagesource, its not there. same with waiting, the javascript processed already, the generated html code is already in his place. and please be patient with me when i mixed something up. im not a web developer, i just started to work on a project and now im stuck. i learned how to use jsoup, i recognized i need to use selenium (btw, i use the firefoxdriver. its fast enough for my purposes and i like to watch what happens when i execute code), i read about javascript but now i really dont have a clue how to proceed.

i didn't add any code, because basically i only use the simple known methods to get the page, click some buttons and get the pagesource. i can add code if it helps.

thanks in advance

stackoverfl000
  • 11
  • 3
  • maybe help: http://stackoverflow.com/questions/16073626/how-can-i-get-html-content-written-by-javascript-with-selenium-python – DevOps85 Nov 04 '14 at 16:51
  • possible duplicate of [Get HTML Source of WebElement in Selenium WebDriver](http://stackoverflow.com/questions/7263824/get-html-source-of-webelement-in-selenium-webdriver) (The answers there cover how to do it in just about any language that Selenium supports.) – Louis Nov 04 '14 at 17:29
  • don't wait for something already there, you have to wait for something added by the javascript. Please provide an example of what you're trying to automate. – user2272115 Nov 04 '14 at 17:38
  • so i have to find out what scripts add the code i want to scrape and then execute them, to get their inner/outer html? i simply dont get it why i have to run them by myself. i will add some code tomorrow, also which pages i am talking about. im in a bit of a hurry right now. but thanks for the answers so far, i will read the api to the mentioned methods and see if i can solve my problems – stackoverfl000 Nov 04 '14 at 18:57

0 Answers0