Stress Testing Tutorial For Beginners
Do you want to learn what is stress testing and how to perform stress testing?
Stress testing is an important part of the software development process. It helps to determine how a program will behave when under stress, and can help developers identify where changes are needed in order to increase stability.
This tutorial covers some basic steps for performing a stress test on your application and the following.
What is Stress Testing?
Stress testing is a type of performance testing that introduces the system to extreme conditions i.e providing overload to the system to check whether it can withstand such stress. It applies a huge volume of data and a large number of users above the expected capacity.
The primary goal of stress testing is to measure the error handling capacity of the software. Most software gets designed to work perfectly in an ideal situation, we won’t know the safe usage limit unless we apply stress. Stress testing helps us to determine the exact breaking point when the software exceeds its limit.
Let’s consider an application that can handle only 100 user and 100 GB of data simultaneously. In stress testing we will apply 150 GB with 120 user in the application concurrently and analyze the behavior.
We can find the answers for the following questions:
- What is the exact point where application break?
- Does the application crash?
- How much time does it take to recover?
- Are they any memory /security issues?
Why do we do stress testing
Let’s take a real-life scenario here, where a website or application might need to stress test before going live.
A larger number of users would be logging into the system concurrently, searching for products, making payments, etc in the website in high volume when a sale is announced in ecommerce website like Amazon or Flipkart.
Such high traffic conditions are common nowadays. If the system fails due to overload in such situations, these sites would lose money and reputation.
- Stress testing verifies the robustness and reliability of the software.
- Stress testing helps us to prepare and prevent system failures.
- Stress testing identifies the security vulnerabilities that might appear during peak conditions.
- Stress testing finds out the recoverability and error handling capabilities of the software.
- Stress testing identifies memory leak issues and sees how the data saving mechanism works during a crash.
- Stress testing helps us to find whether the system gets into a mutual dependency/deadlock situation, any issues related to request priorities.
Types of Stress Testing
Stress Testing covers different aspects of the application through 5 different types of stress testing.
- Distributed Stress Testing
- Application Stress Testing
- Transactional Stress Testing
- Systemic Stress Testing
- Exploratory Stress Testing
#1. Distributed Stress Testing
In this type of stress testing, we test a distributed system with a server connected with multiple client systems.
We apply stress to all client systems and track the status of each client.
Let’s say one or more clients in the distributed systems fail to communicate with the server.
Investigation on that client system would start, debugging possible issues to resolve.
#2. Application Stress Testing
In this type of stress testing, you have to focus on critical performance issues, uncover issues related to data such as bottleneck, data blockage, locks, etc when the application is under stress.
#3. Transactional Stress Testing
In this type of stress testing, you apply stress to one or more transactions between two or more applications.
Transactional stress test works at a transactional level with load volume that exceeds the expected capacity that the system can handle.
As it deals with inter-component interactions, it is useful for tuning, optimizing and finding error conditions at the specific component level.
#4. Systemic Stress Testing
In this type of stress testing, we apply stress or extreme load to several applications running on the same system.
The objective of systematic stress testing is to identify an issue in conditions where different applications block one another and compete for system’s resources like memory, processor, cycles, disk space, and network bandwidth.
#5. Exploratory Stress Testing
In this type of stress testing, we subject the system, application or any components of the software to extreme conditions which are unlikely to happen.
Some samples of how exploratory stress testing scenario:
- All users log in to the application simultaneously.
- Shutting down the database which results in the inability to access data from the system.
- Entering a large volume of input to the database at one go.
Goals of Stress Testing
- To determine the reliability and stability of the software.
- To optimize the system for preventing breakdown.
- To reveal issues that appears only when the system receives high stress.
- To check whether error messages get displayed when subjecting the system to huge traffic.
- To analyze posts crash reports to understand the behavior of the software after failure
- To ensure that the software is not compromised on data integrity, security etc during failures.
- To check whether the application recovers after a failure without intervention.
- To plan scalability and resource requirements correctly.
Strategy for Stress Testing
Test strategy we should follow for stress testing includes the following processes.
- Create a testing environment: Identify the necessary hardware, software, tools and network configuration to develop the test environment.
- Set acceptance criteria for performance: Decide on the success criteria for the stress test, categories and identify the metrics that used for measuring the performance of the software.
- Plan and design the stress test: Identify test scenarios by creating, planning and designing stress tests for the software.
- Configure the test environment: The test environment is set up with the required tools and resources needed for the execution of the plan.
- Implement stress test design: Use the best practices in the industry for developing, designing and implementing stress tests.
- Execute the stress tests: Execute stress testing and monitor the software by collecting and validating the test data and results.
- Analyze the results: Before sharing the results to the rest of the team, make sure that metrics are within acceptable limits, there is no violation in the threshold and enough data is collected during testing.
Stress Testing for Mobile Apps
With the increase in popularity of smart phones, users tend to lean towards mobile applications rather than browsers. So native mobile applications must undergo stress testing.
Follow these points when stress testing mobile:
- Try Monkey stress testing– check the system when constantly changing device orientation, frequent network interruptions, vigorously scrolls up and down etc.
- Use the application with multiple other applications running in the background like music, downloads, app updates etc. simultaneously.
- Focus on interface pages that have a lot of images, video or other high data requiring pages and stress tests on that.
- Identify the most used functionality of the application and stress test them.
- Don’t use emulators or simulators, they fail to show real results.
- Avoid testing in WiFi networks, instead use mobile networks.
- While testing use medium and low-end phones.
- Beta test your application with real users.
How to perform Stress Testing
Step #1: Plan the Stress Test
Collect the required data, analyze the system and come up with a goal for this stress test.
Key scenarios: We have find out frequently used functionalities, that users would access all the time.
Workload: We have to clearly define the amount of load and the number of user required, i.e calculate the peak load and plan on how much more would you will apply for stress testing.
Metrics: Depending on platform in which the application works , metrices change. So decide on which mertics is applicable along with the acceptance limit for them.
Step #2: Create Automation Scripts
For test automation, write scripts and generate test data for each scenario.
Step #3:Execute Scripts
Now run those scripts and store the corresponding results.
Step #4: Analyze results
Examine the test results and identify defects in the system.
Step #5: Tweak and Optimize
Based on the findings from the analysis, now we can focus on fine-tuning the software by changing the configuration and optimizing the code to meet desired results.
Load Testing Vs Stress Testing
LOAD TESTING | STRESS TESTING |
---|---|
It is a subset of performance testing | It is a subset of performance testing |
Goal of load testing is to identify the upper limit of the system, set SLA of the app and check how the system handles heavy load | Goal of stress testing is to find how the system behaves under extreme loads and how it recovers from failure |
Load limit is a threshold of a break | Load limit is above the threshold of a break |
The attributes which are checked in a load testing are peak performance, server throughput, response time under various load levels, load balancing requirements etc. | The attributes which are checked in a stress testing are stability response time, bandwidth capacity etc., |
Load Testing is executed to test the performance of the system under extreme load. | Stress Testing is executed to test the robustness of the system under extreme load. |
It helps us understand how the system behaves under expected load. | It helps us understand how the system behaves in extreme load that exceeds the expected capacity. |
Here the test data will be huge number of users. | Here the test data will be huge number of users and huge volume of data. |
It is used to find out the upper limit of the system's capacity. | It is used to find out the performance and recoverability of the system when it is subject to load that exceeds the upper limit of the system's capacity. |
Load tests help the tester to identify the bottleneck and also help them to understand the root cause for such issues. | Stress tests help the tester to identify memory leaks, slowdowns, security issues, and data corruption. |
Metrics for Stress Testing
Depending on the application under test, the metrics used for stress testing will vary. Some commonly used metrics are :
Measure Scalability & Performance: Response times: It measures the amount of time taken to receive a response after a request is being sent during stress testing.
Pages per Second: It measures how many pages gets requested/read per second.
Throughput: It measures how much data is being sent/received during the stress test.
Rounds: It measures the number of times test scenarios planned versus the number of times a client executes them in stress testing.
Application Response:
Hit time: It is the average time to restore an image or a page when the application is under test.
Time to the first byte: It is the time taken to return the first byte of data or information during a stress test.
Page time: It is the time taken to retrieve all the information on a page when the application is under stress.
Failures:
Failed Connections: It measures the number of connections refused by the client system during a stress test.
Failed Rounds: It measures the number of rounds it fails.Failed Hits: It measures the number of failed attempts done by the system
Examples of Real-World Stress Testing Scenarios
- Ecommerce sites would have a huge surge in activity during mega sales.
- When University websites publish exam results will have multiple users checking results simultaneously .
- Ticket sales open up for a popular movie, multiple users would be booking tickets.
- Millions of users watch viral videos on YouTube.
- News or some important event that increases the traffic for news websites or blogs.
- Finance based applications or websites would have huge traffic when there is a change in stock markets.
- Music streaming apps would have a massive amount of users when a popular artist releases a new album.
Ways to Improve Your Stress Testing Strategy
- Identify the right scenarios: Focus on the most commonly used user workflows. Ex: Checkout process for eCommerce.
- Consider network speed: Most users connect through mobile networks, we should take into account the average user’s network speed while testing.
- Test different types of users: An application with different user roles like a customer, delivery staff, vendor, administrator, etc. Workflows for each role are different, one issue in one of the user roles might affect the other.
- Make it part of your process: Before an expected event like holiday sales, campaign, product launch, etc developers plan stress testing. The best way to integrate it into your process is to incorporate them into your continuous integration and deployment processes.
- Create a plan to assess the results: We must assess the test results and report the issue to the developers. A well-defined plan for quickly fixing the issue before the application is scheduled to deploy should be in place to ensure a smooth process.
Sample Test Cases
As stress testing is non-functional testing, we automate the process using performance testing tools like – Apache JMeter. These tools can provide the application with a huge volume of data, load the application with a large number of users that access the different functionalities of the application.
Some sample test cases:
- Check when the threshold limit gets exceeded and the system crashes.
- Check whether the system can recover after a crash without any intervention.
- Check whether the proper message gets displayed when the system crashes.
- Check how the system responds under stress for various combinations of RAM, processor and network etc.
- Check whether the user can perform the same action when the system exceeds the expected limit.
- Check whether the system is getting into a priority issue when a huge number of users request the same action.
- Check the response time of the system under pressure.
- Check the performance of the system in a very slow network.
- Check all the above test cases for a server that has more than one application running on it to check if the other application gets affected etc.
Conclusion
Stress testing is an essential and beneficial process that provides insight into a variety of metrics tied to potential system damage, including Memory consumption, Processor utilization, Network usage, Disk utilization, Failed/succeeded transactions, Deadlocks, Contentions, Thread allocation etc.
Therefore it is necessary for us as an SDET to perform stress testing to handle such load and fix those issues.