Endurance Testing Guide: How To Perform Endurance Testing

This blog post will cover what endurance testing is, how to perform it and the following

What is Endurance Testing?

Endurance testing comes under non-functional testing, it checks the performance of the system under a specific load condition over an extended period. We can evaluate the system behaviour to ensure that the software is capable of handling extended load without any deterioration of response. It is also known as Soak testing.

Here, we will be conducting endurance testing for a longer period of time depending on the requirements, it can be 12 hours or 72 hours or even more. That is what makes endurance testing different from other types of performance testing like stress or load testing, it monitors the stability of the system over a prolonged period of time.

It is a non-functional type of testing, so we use automation tools like Apache Jmeter, LoadRunner, Appavance IQ, etc.
As it has to run for a long period of time without interruption, we can schedule it during the weekends.
Major issues found in endurance testing are memory leak, connectivity issue, response time degradation, etc.
In endurance testing, we closely monitored memory utilization and performance degradation to identify potential leaks and weak spots.
The optimal time to start endurance testing would be before deployment and after load testing.

Don’t miss: Functional Testing vs Nonfunctional Testing

Why do we do Endurance Testing?

When our application is being used by the customer, there will be a gradual memory leak. In the short term, this won’t affect the system but over time log files grow causing insufficient disk space, and the garbage cleanup process might increase the maintenance cost.

Let’s take an example to understand this. A gaming website creates a lot of server files for a session when the user plays any games on it. Files get deleted once the session ends. These websites may perform well under high load for a short period. But if the load continues to be in a longer session, more files get created, memory might be exhausted or even leak. This may lead to server crashes and impact users.

Endurance testing makes sure that the software application is efficient enough to handle such increased load without response time failure.
It checks whether there is any memory leak in such extreme conditions.
It helps us to assure the robustness and stability of the product.
It monitors how the system performs under prolonged usage.
It can identify database connectivity issues as well.
It checks whether file handlers are being recycled.
It makes sure that log cycling is appropriate.

How to perform Endurance Testing?

Before getting into endurance testing, we have to go through both business requirements and system requirements to see if we need endurance testing. If we have conducted the endurance testing already and we are just adding a new feature to the software, then it might not be required Because response time might be degraded as a result of prolonged testing. But if it’s a new application, it is necessary.

Note that we should execute Endurance testing in an organized systematic and planned way, not ad hoc. So it is usually done by performance testing experts who know how to create and execute a comprehensive test plan.

We can perform endurance testing by following the below steps:

Step #1: Test planning

We start to create a test plan, finalize the resource requirement, cost and estimated time to finish the testing. Along with it, we decide what testing methods we are going to use, manual or automation or both, strategies, tools, how long we are going to perform the test we need.

Other requirements like hardware, software, network and other items required for performing endurance testing are listed down.

Don’t miss: Test Strategy vs Test Plan

Step #2: Test case Design

In this step, we create, design, review and finalize test cases. We build out a test strategy based on this step. Also, we determine the time frame, breakpoint, load, stress that we apply to the system during endurance testing.

Step #3: Testing environment

In this step, we create the testing environment, based on the test plan we gather all the requirements like hardware, software, tools, operating system, network etc. Our goal here is to create an environment that would be similar to the production environment

Step #4: Test Execution

In this step, we perform the execution, if you get any bugs during testing you can note them down and create a bug report. We have seen whether the performance is consistent over time, any external interface is impacting the application, any minor issues not yet addressed etc.

Step #5: Closure

Here, the executing comes to an end when the test cycle meets the exit criteria. We would create a report that contains identified issues, findings and a final report on the application.

What to monitor in Endurance Testing?

Using the usage statistics gathered and the everyday load of the site we determine the peak load. With these historical data as our reference, we can calculate the average load that we are going to apply to the system during testing. If it’s a new application, data may not be available then 50%, 60% or 70 % of load can be considered.

Memory Leak: We have to check if there is any memory leak in the software application, it can often crash your application or operating system.

Normal response time: We have to monitor the response time of the software, as prolonged use can impact the system performance, which leads to less efficient response time.

Connection closure between different layers of the system: We have to see if different layers of the system’s module get closed properly. If it is not closed successfully, it may stall some or all modules in the application.

Successful closure of Test Database: We have to look at whether the database closes properly, else it might lead to our application crashing.

Response time degradation: We can detect response time degradation, this will help the developers fix the problem and provide a reliable system.

Advantages of Endurance Testing

It helps us to find bugs related to a memory leak.
It ensures the robustness of the software application.
There is an added benefit of building a better infrastructure, once we identify bugs and fix them.
It helps us find performance-related issues and check’s the software’s durability.
It checks the sustainability of the system over time.
It reduces maintenance costs as we identify these issues early on.
It uncovers issues that we might not have seen while conducting other performance tests.
It can give us the exact value of how much load the system can sustain for how long.
It gives us confidence in the application and provides customer satisfaction.

Disadvantages of Endurance Testing

Endurance testing can be time-consuming.
We need experts with automation knowledge and automation tools as it can’t be done manually.
It can cost a lot as we have to run the system for a long time.
While conducting endurance testing, we have to isolate the test environment from the live as it might affect the performance of the live. This results in us building another infrastructure that is not related to the current one.
We often don’t know how much load is worth applying.
If we over-stress the system, it may result in permanent data loss or corruption
The end-user might notice those unhandled exceptions.

Realtime example of Endurance Testing

In an ideal scenario, Endurance testing should run as long as possible with a different set of loads to understand where the system gets unstable. But we can only work with the given tools, environment & resources with a timeline.

Here is a real-time example of endurance testing.

Let’s say a lockdown is announced and the customer cannot visit the physical bank location. So a large number of transactions will happen online. The banking website and mobile application should be capable enough to handle such a huge set of transactions for a long period. We have to make sure that the system can function smoothly without any data leak or interruptions.

To avoid any last-minute failure we can perform endurance testing with our banking application.

We can start by assigning 1000 transactions to get processed in 1 hr and see how the system handles such requests. Later we can increase the transaction to 2000 in 1 hour. We might increase the load and duration of the test based on the given requirements.

Another example would be search engines like Google. There are millions of people who would use it around the world. It has different scenarios like search bar being idle for a long time, user exiting or leaving unattended in the middle of the search, vigorously searching for information and many more. Other examples would be social media, email service, gaming applications, etc which will have millions of users accessing and using them for a long time.

So we have to make sure that our application has a high response time, good connectivity in database and system layers without compromising on memory. Endurance testing might be time-consuming and requires a lot of resources but if it’s done right, we can ensure the high-end performance of our system.

Related posts:

Endurance Testing Guide: How To Perform Endurance Testing

What is Endurance Testing?

Why do we do Endurance Testing?