Selenium Java - Fetching page source via JavascriptExecutor

Question

I want to read the page's source code after it has changed due to some ajax requests. Since I'm running out of ideas, I tried JavaScript, but the executeScript() function always returns null. Any help would be much appreciated.

I use the FirefoxDriver. My code looks as follows:

Object test = ((JavascriptExecutor) firefox).executeScript("function getHTML(){ return document.body.innerHTML;} getHTML();"); System.out.println(test);

alecxe · Accepted Answer · 2015-02-05 22:21:48Z

4

You need to have something returned from the script:

Object test = ((JavascriptExecutor) firefox).executeScript("return document.body.innerHTML;");

answered Feb 5, 2015 at 22:21

alecxe

475k127 gold badges1.1k silver badges1.2k bronze badges

I thought calling my function via getHTML() would be enough. But your solution works, so I am very grateful. Cheers! :)
– MightyMalcolm
CommentedFeb 5, 2015 at 22:26
@alecxe Why OPS code doen't work. I can easily test that on Chrome and works
– Saifur
CommentedFeb 5, 2015 at 22:29
@Saifur well, adding the return could also do the trick: function getHTML(){ return document.body.innerHTML;} return getHTML();.
– alecxe
CommentedFeb 5, 2015 at 22:30
1
@Saifur yup, tested it using the python bindings: without the return I get nothing, with - I get the innerHTML of the body. Thanks.
– alecxe
CommentedFeb 5, 2015 at 22:32
@alecxe hmm, it seems i was a bit too fast yesterday and my question isn't answered after all. the JavaScript execution now works, but it gives me the stale sourcecode that does not reflect the ajax changes. My issue seems to be similiar to this one that you answered as well, but it relies on the same method.
– MightyMalcolm
CommentedFeb 6, 2015 at 9:49

Add a comment |

Saifur · Accepted Answer · 2015-02-07 06:03:22Z

I am assuming the dom is not in ready state and you are trying to fetch is too fast. I am modifying alecxe's answer a little bit to add additional explicit wait that will make sure the dom is in ready state. This program will wait until 10 s to make sure the dom is in ready state.

Wait<WebDriver> wait = new WebDriverWait(driver,10); wait.until(new Function<WebDriver, Boolean>() { public Boolean apply(WebDriver driver) { return ((JavascriptExecutor) driver).executeScript("return document.readyState;").equals("complete"); } }); Object test = ((JavascriptExecutor) driver).executeScript("return document.body.innerHTML;");

really appreciate your efforts, but they seem to make no difference. I am checking for some attribute values and the fun thing is, that driver.findElement() returns them correctly whilst yours and alecxe's does not. Still, every findElement() call takes about 0.1 secs and i have to check more than 10 elements in a time-critical moment. My code looks as follows, maybe i've done sth wrong? — MightyMalcolm, CommentedFeb 7, 2015 at 9:28
Object test = stackOverflowMethod(firefox); Document doc = Jsoup.parse(test.toString()); debugger.println(doc.getElementById("input_field1").attr("value")); debugger.println(firefox.findElement(By.xpath(XPaths.INPUT_FIELD_1).getAttribute("value")); The stackOverfflowMethod() obviously is 100% the code you posted. *EDIT: sry i am too stupid to use the code tag in comments appropriately it seems -.- — MightyMalcolm, CommentedFeb 7, 2015 at 9:37

Collectives™ on Stack Overflow

Selenium Java - Fetching page source via JavascriptExecutor

2 Answers 2

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Linked

Related