Configuration Failure in TestNG - a Case Study

How I Overcame Configuration Failures in Test Automation

Introduction:

In automation projects especially in the TestNG framework, I have faced many challenges, but one that really needed more attention was configuration failure during parallel test executions. It was not just a small bug; it had the potential to break the entire test flow.

In this blog, I want to share my personal experience dealing with configuration failure issue, how the root cause was discovered, and the possible ways to solve the issue.

The Frustration of Configuration Failures:

When I was running the test cases, everything seemed fine until that some test cases were being skipped particularly when the test suite was executed in parallel mode at class level. It was very difficult to understand why the test cases are skipped even though the logic was correct. After digging in, the setup methods and tear down methods like @BeforeSuite, @BeforeTest, and @AfterMethod were failing and causing the entire dependent test flow to collapse.

It was frustrating because the logic of the test is correct, but the failures were happening before the tests even got a chance to run, and also it took longer to find the root cause than initially anticipated.

What was Going Wrong?

With multiple attempts of debugging and analysis into the logs and code, some of the key issues identified that causes configuration failures are

• Thread Safety Problems:

  The lack of thread safety in WebDriver instances and Excel readers. Using shared resources without any isolation.

  
  

• Improper Driver Factory Design:

  If the driver factory is not designed for parallel execution, it leads to configuration failure.

  
  

• Unhandled Alerts:

  An alert can appear unexpectedly on the web application during test execution. If this alert is not handled explicitly in the automation script, it will throw an UnhandledAlertException and in-turn cause configuration failures.

  
  

•  Improper Setup Flow:

  When the test case flow is not proper especially when the setup methods failed, the rest of the test cases skipped which will cause configuration failure.

  
  

What Actually Worked?

A few changes in the framework combined with proper setup and teardown methods, resolved the issue. By ensuring the proper test case flow with proper exception handling, the tests started executing reliably even in parallel mode.

•  ThreadLocal Workbooks:

  Implemented ThreadLocal Excel reader for data driven testing. This ensured each thread had its own isolated copy of the objects, eliminating the chance of clashes during parallel runs. For example,

private static final ThreadLocal<Workbook> threadLocalWorkbook = new ThreadLocal<> ();

• Combining ThreadLocal and Map/List:

  When using data providers, combining ThreadLocal with thread safe data structures like ConcurrentHashMap helped prevent data conflicts and collisions when multiple threads tried to access shared data simultaneously.

  
  

• Thread safe Driver Factory:

  Designed the Driver Factory to ensure it generated a thread safe WebDriver Instance for each test. This included proper initialization and tear down to avoid browser leaks. For example,

protected static final  ThreadLocal<WebDriver> webdriver = new ThreadLocal<> ();

• Set up the Test Case Flow:

  Designed the test methods and the TestNG annotations such as @BeforeSuite, @AfterSuite, @BeforeClass, @AfterClass, @BeforeTest, @AfterTest, @BeforeMethod, @AfterMethod properly in a way that doesn't affect other test cases, preventing tests from being skipped or failing due to set up issues. This ensures the stable and consistent flow of execution in parallel mode.

  
  

• Alert Handling:

  Unhandled alerts like JavaScript alert (), confirm (), prompt () or popups can cause interruption in test execution and trigger configuration failure. These alerts become problematic when they appear during setup or tear down phases like @BeforeClass, @AfterClass, @BeforeMethod, @AfterMethod and so on. If these methods fail due to unhandled alerts, TestNG does not execute the associated test methods instead it skips them make the run as a configuration failure. Few methods to handle alerts are

Alert Type	Example	Selenium Handling
Simple Alert	alert("Success")	Only OK button
Confirmation Alert	confirm ("Do you want to continue")	OK and Cancel buttons
Prompt Alert	prompt ("Enter your name")	Input field and OK/Cancel buttons
Unexpected Alert	Appears Unpredictably	can break test flow if not handled

1. Handling Simple Alert

Alert alert  = driver.switchTo(). alert ();
alert.accept();

2. Handling Confirmation Alert

Alert alert  = driver.switchTo(). alert ();
String alertText = alert.getText();
alert.dismiss();

3. Handling Prompt Alert

Alert alert  = driver.switchTo(). alert ();
alert.sendkeys("Ajith Kumar");
alert.accept();

4. Handling Unexpected Alert

A try catch block can be used to handle unexpected alerts by capturing and dismissing them without

crashing, which prevents configuration failures and ensures tests are not skipped unnecessarily.

try {
Alert alert  = driver.switchTo(). alert ();
alert.dismiss();
} catch (NoAlertPresentException e) {
loggerload.info ("No alert is present");
}

If an alert is expected after performing an action such as clicking a button or submitting a form

WebDriverWait can be used to wait explicitly for the alert to appear before attempting to interact with it.

This ensures the alert is properly handled without throwing an UnhandledAlertException or the test

skipped before execution.

WebDriverWait wait = new WebDriverWait (driver, Duration.ofSeconds (15));
wait.until (ExpectedConditions.alertIsPresent());	
Alert alert  = driver.switchTo(). alert ();
alert.accept();

What I Learned?

Configuration failure was more than just fixing a bug. It may happen because of any of the conditions as discussed. It makes the entire suite unreliable even if the test scripts are perfect and the logic is correct. Invest time in setting up the framework with robust methods, clear exception handling, thread safe design to ensure reliable and consistent test execution.

Conclusion

Facing configuration failure during that phase was frustrating but it forced me to dig deeper and explore more, which helped me to learn and understand the framework and the concept better.

If you are getting configuration failure with skipped tests, check the configuration methods and thread safety first. It might save your days of debugging.

"Every skipped test tells a story - make sure yours has a solution."