How To Run Failed Test Cases Using TestNG In Selenium WebDriver

Run Failed Test Cases Using TestNG in Selenium WebDriver:

At times, test cases may fail while running automated test scripts. The reason may be anything (say, Network issue, System issue or browser issue) but as an automation tester, you need to execute the test scripts again. Here is a solution to run failed test cases using TestNG in Selenium.

Don’t you want to take a screenshot on failure and attach that in your report?

Yeah, as a tester we need to capture the failed steps. This post guides you on “How To Capture Screenshot and Insert it in Extent Reports”

We could execute the failed test cases in two ways.

Case 1: Execute failed test cases using TestNG in Selenium – By using “testng-failed.xml”

Steps To follow:

After the first run of an automated test run. Right click on Project – Click on Refresh
A folder will be generated named “test-output” folder. Inside “test-output” folder, you could find “testng-failed.xml”
Run “testng-failed.xml” to execute the failed test cases again.

Case 2: Execute failed test cases using TestNG in Selenium – By Implementing TestNG IRetryAnalyzer.

Create a class to implement IRetryAnalyzer. Here I am creating a class (say, RetryFailedTestCases) and implementing IRetryAnalyzer.

RetryFailedTestCases implements IRetryAnalyzer:

package softwareTestingMaterial;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryFailedTestCases implements IRetryAnalyzer {
    private int retryCnt = 0;
    //You could mentioned maxRetryCnt (Maximiun Retry Count) as per your requirement. Here I took 2, If any failed testcases then it runs two times
    private int maxRetryCnt = 2;
    
    //This method will be called everytime a test fails. It will return TRUE if a test fails and need to be retried, else it returns FALSE
    public boolean retry(ITestResult result) {
        if (retryCnt < maxRetryCnt) {
            System.out.println("Retrying " + result.getName() + " again and the count is " + (retryCnt+1));
            retryCnt++;
            return true;
        }
        return false;
    }
   
}

Let’s create another class ‘RetryListenerClass’ by Implementing ‘IAnnotationTransaformer’ interface. transform method is called for every test during test run. A simple implementation of this ‘IAnnotationTransformer’ interface can help us set the ‘setRetryAnalyzer’ for ‘ITestAnnotation’. Add the above class name (RetryFailedTestCases.class) in the below program. This interface does its work in run time by adding annotation to the test methods.

RetryListenerClass implements IAnnotationTransformer:

package softwareTestingMaterial;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;

import org.testng.IAnnotationTransformer;
import org.testng.IRetryAnalyzer;
import org.testng.annotations.ITestAnnotation;

public class RetryListenerClass implements IAnnotationTransformer {

	@Override
	public void transform(ITestAnnotation testannotation, Class testClass, Constructor testConstructor, Method testMethod)	{
		IRetryAnalyzer retry = testannotation.getRetryAnalyzer();

		if (retry == null)	{
			testannotation.setRetryAnalyzer(RetryFailedTestCases.class);
		}

	}
}

Let us see the example by executing simple tests below. Here I took two test cases say Test1 and Test2.

Testcase 1:

package softwareTestingMaterial;

import org.testng.Assert;
import org.testng.annotations.Test;

public class Test1 {

	@Test
	public void test1(){
		System.out.println("Test 1");
		Assert.assertTrue(true);
		
	}
}

Testcase 2:

package softwareTestingMaterial;

import org.testng.Assert;
import org.testng.annotations.Test;

public class Test2 {
	@Test
	public void test2(){
		System.out.println("Test 2");
		Assert.assertTrue(false);
	}
}

As per the lines of code in Test2, it will fail. So it (Test2) will be executed 2 times (we took the maxRetryCnt as 2) in Retry Class. First lets include below mentioned Listener to testng.xml file. Below mentioned syntax is to add Listener for RetryListnereClass

<listeners>
        <listener class-name="softwareTestingMaterial.RetryListenerClass"/>
</listeners>

Final testng.xml file should looks like below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="My Suite">

<listeners>
        <listener class-name="softwareTestingMaterial.RetryListenerClass"/>
  </listeners>

	<test name="Test1">
	
		<classes>
			<class name="softwareTestingMaterial.Test1" />
		</classes>
	</test> <!-- Test -->
	<test name="Test2">
	
		<classes>
			<class name="softwareTestingMaterial.Test2" />
		</classes>
	</test> <!-- Test -->

</suite> <!-- Suite -->

Execute the testng.xml. Here is the output which I got. You could see in the below mentioned result that the Test 2 is executed three times as we have mentioned ‘maxRetryCnt = 2’. Even though we have just 2 tests, we could find total test runs are 4 in the result.

[TestNG] Running:
  D:\Selenium\workspace\SeleniumProject\testng.xml

Test 1
Test 2
Retrying test2 again and the count is 1
Test 2
Retrying test2 again and the count is 2
Test 2

===============================================
Everjobs Suite
Total tests run: 4, Failures: 1, Skips: 2
===============================================

This way we could run failed test cases using TestNG in Selenium.