Discussion:
Refresh Selenium WebDriver DOM tree - correcting a lot of NoSuchElement, StaleReference exceptions
Peter Merkač
2013-05-27 09:24:35 UTC
Permalink
Hi Selenium friends,

With my coworkers we are currently working on a project for automating GUI
tests execution for insurance web applications. Lately we had a lot of
instability problems with our Selenium test automation kit (a.k.a. SeTAK),
which is built on top of Selenium WebDriver. Those Selenium errors were
related to following three areas:
- switching of frames and consequently (NoSuchElementException)
- generally elements were not found (when searching by id, name or xPath)
although they were visible on the web-page
- stale reference exceptions also occurred again with elements visible on
the web-page

Than we came across a solution to retry to execute particular selenium
commands like in the below link

http://stackoverflow.com/questions/4846454/selenium-webdriver-staleelementreferenceexception

While some of the above mentioned errors were corrected as suggested by the
retry solution, a situation, that particular DOM element was NOT present in
the Selenium WebElement tree, although VISIBLE on the web-page of target
web-application, has NOT been tackled. From our experience such situations
were undeterministic, occured randomly and more often on the DOM elements,
which had some JavaScript events e.g "onchange", ... in-behind.
We came to the conclusion that there MUST be some bug in refreshing the
tree of WebElements as maintained by the selenium WebDriver, which points
to actual DOM three of a target web-page. Having this in mind we
implemented a simple but VERY effective solution (tested on our test
environment multiple times), which has been driven by the below
Stackoverflow force DOM refresh without page reload. Our requirement is,
that with DOM refresh no page reload is done.

http://stackoverflow.com/questions/8840580/force-dom-redraw-refresh-on-chrome-mac
From our experience with above described errors we suggest instead of
retrying to interpret selenium commands (or additionally to retry solution)
to REFRESH DOM tree of a web-page with javascript as suggested in the above
link and than retry to execute web driver commands.

@Override
public void refreshDOM(String elementId) throws WebDriverException {
switchFrame(getActiveFrameId(), 5);
JavascriptExecutor jsExecutor = (JavascriptExecutor) getWebDriver();
jsExecutor.executeScript(buildJQueryScriptDerictive());
jsExecutor.executeScript("$('#" + elementId + "').hide().show();");
}

Of course such DOM refresh has to be used with care in sense of
time-execution impact only when a particular undetermined exception occurs
like in the case below.

public IWebTestResult interpretPredicateWithRetry(IInterpretationContext
interpretationContext)
throws WebDriverException {
try {
interpretAndLogPredicate(interpretationContext, logMessage,
false, 1);
}
catch (NoSuchElementException e1) {

interpretationContext.getPredicateHandler().refreshDOM("UI-MAIN_PANEL");
// Retry to interpret commands

}
catch (StaleElementReferenceException e2) {

interpretationContext.getPredicateHandler().refreshDOM("UI-MAIN_PANEL");
// Retry to interpret commands
}
catch (NullPointerException e3) {

interpretationContext.getPredicateHandler().refreshDOM("UI-MAIN_PANEL");
// Retry to interpret commands
}
catch (TimeoutException e4) {

interpretationContext.getPredicateHandler().refreshDOM("UI-MAIN_PANEL");
// Retry to interpret commands
}
finally {
interpretationContext.setWaitInterpretationHandler(new
DefaultWaitInterpretationHandler());
}
return predicateResult;
}
We are passing in an id of top-parent element, but to fine tune this, an
exact parent element can be found via java-script to decrease performance
impact.

This solution brought a LOT of stability to our selenium gui tests,
therefore we wanted to share it with all of you.

With best regards,
Peter, Toni
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to selenium-users-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/236566e3-8d82-47ba-ab7a-6e7d5b7af3d1%40googlegroups.com?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
Mark Collin
2013-05-28 08:01:21 UTC
Permalink
X-Received: by 10.180.98.226 with SMTP id el2mr761726wib.6.1369730210205;
Tue, 28 May 2013 01:36:50 -0700 (PDT)
X-BeenThere: selenium-users-/***@public.gmane.org
Received: by 10.181.12.70 with SMTP id eo6ls1022428wid.21.canary; Tue, 28 May
2013 01:36:49 -0700 (PDT)
X-Received: by 10.180.185.115 with SMTP id fb19mr5883981wic.0.1369730209735;
Tue, 28 May 2013 01:36:49 -0700 (PDT)
Received: from beshaba.lazeryattack.com (mail.lazeryattack.com. [81.27.73.110])
by gmr-mx.google.com with ESMTP id ch8si510764wib.3.2013.05.28.01.36.48
for <selenium-users-/***@public.gmane.org>;
Tue, 28 May 2013 01:36:48 -0700 (PDT)
Received-SPF: pass (google.com: best guess record for domain of mark.collin-R3r3+***@public.gmane.org designates 81.27.73.110 as permitted sender) client-ip=81.27.73.110;
Received: from localhost (localhost [127.0.0.1])
by beshaba.lazeryattack.com (Postfix) with ESMTP id E49A944AF4
for <selenium-users-/***@public.gmane.org>; Tue, 28 May 2013 09:36:47 +0100 (BST)
X-Virus-Scanned: Debian amavisd-new at beshaba.ardescosolutions.com
Received: from beshaba.lazeryattack.com ([127.0.0.1])
by localhost (beshaba.lazeryattack.com [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id gAZfi3vVUdKW for <selenium-users-/***@public.gmane.org>;
Tue, 28 May 2013 09:36:32 +0100 (BST)
Received: by beshaba.lazeryattack.com (Postfix, from userid 106)
id CBA1944AEB; Tue, 28 May 2013 09:01:02 +0100 (BST)
Received: from Mili-Collins-MacBook-Air-124.local (host-80-194-146-206.static.cable.virginmedia.com [80.194.146.206])
by mail.lazeryattack.com (Postfix) with ESMTPSA id 4798344AD5
for <selenium-users-/***@public.gmane.org>; Tue, 28 May 2013 09:01:01 +0100 (BST)
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130509 Thunderbird/17.0.6
In-Reply-To: <236566e3-8d82-47ba-ab7a-6e7d5b7af3d1-/***@public.gmane.org>
X-Original-Sender: mark.collin-R3r3+***@public.gmane.org
X-Original-Authentication-Results: gmr-mx.google.com; spf=pass
(google.com: best guess record for domain of mark.collin-R3r3+***@public.gmane.org
designates 81.27.73.110 as permitted sender) smtp.mail=mark.collin-R3r3+***@public.gmane.org
Precedence: list
Mailing-list: list selenium-users-/***@public.gmane.org; contact selenium-users+owners-/***@public.gmane.org
List-ID: <selenium-users.googlegroups.com>
X-Google-Group-Id: 715004705585
List-Post: <http://groups.google.com/group/selenium-users/post?hl=en-US>, <mailto:selenium-users-/***@public.gmane.org>
List-Help: <http://groups.google.com/support/?hl=en-US>, <mailto:selenium-users+help-/***@public.gmane.org>
List-Archive: <http://groups.google.com/group/selenium-users?hl=en-US>
Sender: selenium-users-/***@public.gmane.org
List-Subscribe: <http://groups.google.com/group/selenium-users/subscribe?hl=en-US>,
<mailto:selenium-users+subscribe-/***@public.gmane.org>
List-Unsubscribe: <http://groups.google.com/group/selenium-users/subscribe?hl=en-US>,
<mailto:googlegroups-manage+715004705585+unsubscribe-/***@public.gmane.org>
Archived-At: <http://permalink.gmane.org/gmane.comp.web.selenium.user/12682>

I hate to say it but your problems are indicative of badly constructed
test cases and/or lack of understanding of how the website you are
automating works.

NoSuchElementException's are caused because when selenium looked at the
page the element wasn't there, remember that Selenium is fast, much
faster than a human so it is quite able to check the DOM *before* all
JavaScript rendering has taken place. The solution to these issues is
to use an explicit wait and wait for the element to appear.

If you don't want to use explicit waits and you want a bit of a hacky
solution that will work in most cases try using an implicit wait,
whatever you do don't mix implicit and explicit waits, things will start
going wrong, explicit waits are the recommended solution:

http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#explicit-and-implicit-waits

StaleElementReferenceException's are caused by the DOM being refreshed
after you found an element. Remember that a WebElement is a reference
to a specific element on the page, if the DOM get's refresed this
reference is broken and you need to find the element again to be able to
interact with it. The best way to deal with these in my opinion is to
start using page factories. The @FindBy annotation will ensure that the
element if found every time you use it so you will not suffer from
StaleElementReferenceException's (Well OK there is a slim possibility
that it can occure if the DOM changes in the couple of ms between it
finding the element and doing something with it, but that's a real edge
case.

https://code.google.com/p/selenium/wiki/PageFactory
Post by Peter Merkač
Hi Selenium friends,
With my coworkers we are currently working on a project for automating
GUI tests execution for insurance web applications. Lately we had a
lot of instability problems with our Selenium test automation kit
(a.k.a. SeTAK), which is built on top of Selenium WebDriver. Those
- switching of frames and consequently (NoSuchElementException)
- generally elements were not found (when searching by id, name or
xPath) although they were visible on the web-page
- stale reference exceptions also occurred again with elements visible
on the web-page
Than we came across a solution to retry to execute particular selenium
commands like in the below link
http://stackoverflow.com/questions/4846454/selenium-webdriver-staleelementreferenceexception
While some of the above mentioned errors were corrected as suggested
by the retry solution, a situation, that particular DOM element was
NOT present in the Selenium WebElement tree, although VISIBLE on the
web-page of target web-application, has NOT been tackled. From our
experience such situations were undeterministic, occured randomly and
more often on the DOM elements, which had some JavaScript events e.g
"onchange", ... in-behind.
We came to the conclusion that there MUST be some bug in refreshing
the tree of WebElements as maintained by the selenium WebDriver, which
points to actual DOM three of a target web-page. Having this in mind
we implemented a simple but VERY effective solution (tested on our
test environment multiple times), which has been driven by the below
Stackoverflow force DOM refresh without page reload. Our requirement
is, that with DOM refresh no page reload is done.
http://stackoverflow.com/questions/8840580/force-dom-redraw-refresh-on-chrome-mac
From our experience with above described errors we suggest instead of
retrying to interpret selenium commands (or additionally to retry
solution) to REFRESH DOM tree of a web-page with javascript as
suggested in the above link and than retry to execute web driver commands.
@Override
public void refreshDOM(String elementId) throws WebDriverException {
switchFrame(getActiveFrameId(), 5);
JavascriptExecutor jsExecutor = (JavascriptExecutor) getWebDriver();
jsExecutor.executeScript(buildJQueryScriptDerictive());
jsExecutor.executeScript("$('#" + elementId + "').hide().show();");
}
Of course such DOM refresh has to be used with care in sense of
time-execution impact only when a particular undetermined exception
occurs like in the case below.
public IWebTestResult
interpretPredicateWithRetry(IInterpretationContext interpretationContext)
throws WebDriverException {
try {
interpretAndLogPredicate(interpretationContext,
logMessage, false, 1);
}
catch (NoSuchElementException e1) {
interpretationContext.getPredicateHandler().refreshDOM("UI-MAIN_PANEL");
// Retry to interpret commands
}
catch (StaleElementReferenceException e2) {
interpretationContext.getPredicateHandler().refreshDOM("UI-MAIN_PANEL");
// Retry to interpret commands
}
catch (NullPointerException e3) {
interpretationContext.getPredicateHandler().refreshDOM("UI-MAIN_PANEL");
// Retry to interpret commands
}
catch (TimeoutException e4) {
interpretationContext.getPredicateHandler().refreshDOM("UI-MAIN_PANEL");
// Retry to interpret commands
}
finally {
interpretationContext.setWaitInterpretationHandler(new
DefaultWaitInterpretationHandler());
}
return predicateResult;
}
We are passing in an id of top-parent element, but to fine tune this,
an exact parent element can be found via java-script to decrease
performance impact.
This solution brought a LOT of stability to our selenium gui tests,
therefore we wanted to share it with all of you.
With best regards,
Peter, Toni
--
You received this message because you are subscribed to the Google
Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
https://groups.google.com/d/msgid/selenium-users/236566e3-8d82-47ba-ab7a-6e7d5b7af3d1%40googlegroups.com?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to selenium-users-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/51A46451.5010109%40lazeryattack.com?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
Muralisankar A
2014-11-04 13:06:14 UTC
Permalink
import java.util.concurrent.TimeUnit;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Reporter;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;


public class Testcases {

WebDriver driver;
String baseurl="http://192.168.0.190/users/login/";

@BeforeTest
public void setup() {

driver=new FirefoxDriver();
driver.get(baseurl);
driver.manage().window().maximize();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
}
@Test
public void TC8() {

driver.findElement(By.id("id_username")).sendKeys("admin");
driver.findElement(By.id("id_password")).sendKeys("admin");

driver.findElement(By.cssSelector("button,input[type='button']")).click();
Reporter.log("TC101 > Login successfully to crossfraudet");
}

@Test
public void TC9() {

driver.findElement(By.cssSelector(".icon-users"));

}
@Test
public void TC10() {

driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.findElement(By.cssSelector(".icon-users")).click();
driver.findElement(By.cssSelector("#menuList > li.dropdown.open >
ul > li:nth-child(3) > a")).click();
Reporter.log("TC102 > Click on User Profile > Profiles");

}


}

[TestNG] Running:

C:\Users\Murali\AppData\Local\Temp\testng-eclipse-769164162\testng-customsuite.xml

PASSED: TC8
PASSED: TC9
FAILED: TC10
org.openqa.selenium.NoSuchElementException: Unable to locate element:
{"method":"css selector","selector":".icon-users"}
Command duration or timeout: 10.04 seconds
For documentation on this error, please visit:
http://seleniumhq.org/exceptions/no_such_element.html



How can I solve this issue? Anybody please help me
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+***@googlegroups.com.
To post to this group, send email to selenium-***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/617880c8-d244-47ce-a738-9b065cf8876a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
PeterJeffreyGale
2014-11-04 13:11:28 UTC
Permalink
Post by Muralisankar A
How can I solve this issue? Anybody please help me
1. Don't hijack threads
2. Don't use findElement() until just before you are about to use the
elements you are seeking.
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+***@googlegroups.com.
To post to this group, send email to selenium-***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/df50fa26-1005-44ba-bd09-58d91f4c82e9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Muralisankar A
2014-11-04 13:22:12 UTC
Permalink
Hi peter, thanks for your reply.

Please send me a sample code to solve this issue


On Tue, Nov 4, 2014 at 6:41 PM, PeterJeffreyGale <
Post by PeterJeffreyGale
Post by Muralisankar A
How can I solve this issue? Anybody please help me
1. Don't hijack threads
2. Don't use findElement() until just before you are about to use the
elements you are seeking.
--
You received this message because you are subscribed to a topic in the
Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/selenium-users/3cshBBojFrE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
To view this discussion on the web visit
https://groups.google.com/d/msgid/selenium-users/df50fa26-1005-44ba-bd09-58d91f4c82e9%40googlegroups.com
<https://groups.google.com/d/msgid/selenium-users/df50fa26-1005-44ba-bd09-58d91f4c82e9%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+***@googlegroups.com.
To post to this group, send email to selenium-***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAOk%2Bzy9Dz1pSgj_hTohNZ4KWkdJ8fX%2BM3dVr--UapDS6iE1Neg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
PeterJeffreyGale
2014-11-04 13:23:49 UTC
Permalink
You can write you're own code. Just apply the principle suggested.
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+***@googlegroups.com.
To post to this group, send email to selenium-***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/decd397d-a3ef-4ee3-a934-52eaa975de72%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Muralisankar A
2014-11-04 13:26:02 UTC
Permalink
I have been struggling to find the element on the page after logged in to
my application.
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+***@googlegroups.com.
To post to this group, send email to selenium-***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAOk%2Bzy_%3DeYOCEAhOefJQPLexhu-s77bS2Ed7Tr3p%2BBLPBk14cA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Muralisankar A
2014-11-04 13:31:03 UTC
Permalink
It shows the same error if I use the xpath
driver.findElement(By.xpath("/html/body/div/div/section/nav/ul/li[3]/a/i"))
instead of css selector driver.findElement(By.cssSelector(".icon-users"))
Post by Muralisankar A
I have been struggling to find the element on the page after logged in to
my application.
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+***@googlegroups.com.
To post to this group, send email to selenium-***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAOk%2Bzy_CnOuq3R968sijn3Df5GbpL2y6Gioa96paZ5b%2BR3mWZg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Muralisankar A
2014-11-04 16:04:53 UTC
Permalink
@Test
public void TC9() {
WebDriverWait wait = new WebDriverWait(driver, 30);

wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector(".icon-users")));
}

@Test
public void TC10() {
WebDriverWait wait = new WebDriverWait(driver, 30);

wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector(".icon-users")));
driver.findElement(By.cssSelector(".icon-users")).click();
driver.findElement(By.cssSelector("#menuList > li.dropdown.open >
ul > li:nth-child(3) > a")).click();
Reporter.log("TC102 > Click on User Profile > Profiles");
}


PASSED: TC8
PASSED: TC9
FAILED: TC10
org.openqa.selenium.TimeoutException: Timed out after 30 seconds waiting
for visibility of element located by By.selector: .icon-users
Build info: version: '2.43.1', revision: '5163bce', time: '2014-09-10
16:27:33'
System info: host: 'BDMAGFPC009', ip: '192.168.12.118', os.name: 'Windows
7', os.arch: 'x86', os.version: '6.1', java.version: '1.8.0_05'
Driver info: org.openqa.selenium.firefox.FirefoxDriver
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+***@googlegroups.com.
To post to this group, send email to selenium-***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/3c8e2c93-29d9-4d57-b316-dc130128bfea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...