Load Testing vs. Stress Testing
Load and Stress Testing Compared
A load test, by definition, measures the performance of a system under an expected load.
In contrast, a stress test overloads a system in order to find the breaking point.
Examining the Differences:
Load vs. Stress Testing
A load test is a planned test to perform a specified number of requests to a system in order to test the functionality of the system under specific levels of simultaneous requests. A load test ensures that a web system is capable of handling an expected volume of traffic, and therefore is sometimes referred to as volume testing. The goal of a load test is to prove that a system can handle the expected volume with minimal to acceptable performance degradation. The threshold of acceptable performance degradation must be defined by the testers as some value considered acceptable to the end user so that users will not bounce from the site.
A stress test is a test designed to increase the number of simultaneous requests on a system past a point where performance is degraded, possibly even to the point of complete failure. Where a load test will peak out in the number of simultaneous users, a basic stress test will continue to increase load on the system until the resources are overloaded. This pushes the system to a state of potential failure to see how the system handles it and whether the system can perform a graceful recovery.
Within these definitions of a load test and a stress test, we find that they are certainly not completely independent from one another. Often times, when running the upper boundaries of a load test, you may effectively end up running a stress test where you push the system past the limits of available resources. At this point you might start to see failures in a load test identical to the failures typically seen when running a stress test.
When to Choose a Load Test or Stress Test
One difference between a load test and stress test is that you may inject pauses into a load test to simulate real user traffic. With a stress test, you may run as many simultaneous users as fast as possible to generate excessive traffic for a stress test.
The goals of a load test are very different than the goals of a stress test. A load test is performed in order to ensure that a website or web application is capable of handling specific numbers of users at once. Load Testing is often used in the process of capacity planning, to ensure that a system is able to handle growth to specified levels of simultaneous traffic.
A stress test is used to specifically push a system beyond its intended capacity to identify components that begin to slow down, identify bottlenecks in the system, and bring to light possible points of failure.
Commonly Used for Load & Stress Testing:
Establishing Baseline Performance Metrics
Load testing is typically performed as a series of steps where the testing system initiates a quantity of simultaneous users that is known to be supported by the infrastructure. This establishes a baseline set of performance data to reference as the number of simultaneous users is increased throughout the test. This baseline test is sometimes called performance testing. A performance test can help determine a number of different benchmarks such as average connection speed, average latency, and average time to download a file of a fixed size and more.
Once baseline performance values are known, the number of users is increased to a number that may be realistically expected to visit the site during a sample period. The test will often then run at that static number of users for several minutes in order to verify the stability of the website after the system stabilizes at the new load level.
One difference between establishing a baseline performance metric during a load test and a stress test is that the difference between the baseline and peak performance will help determine if you have the proper systems in place to handle peak load, while during a stress test you are more concerned about the point at which the system becomes stressed and possibly even ceases to work properly.
Load or Stress Testing to Identify Web App Bottlenecks
Web based applications typically run in a browser and when programmed properly, due to their asynchronous nature, may handle many hundreds or thousands of simultaneous users. As long as you are generating expected load within the capacity of your system, the response times of the application should stay within generated guidelines. If you push the system beyond these limits you move into the realm of stress testing, purposely causing strain on the system to identify the components that fail. Therefore, any test performed for the purpose of identifying bottlenecks is typically considered a stress test.
Load Testing to Establish Service Level Agreements (SLAs)
Load tests are best performed in a production environment to understand average response times under expected user load. These average response times become the baseline for acceptable SLAs. From here, it is up to you to determine additional thresholds that are considered unacceptable under your SLAs in terms of expected performance for your customers.
Load Testing for Capacity Planning
Generating increased load on a web application can help predict application performance for heavier user load in the future. As long as the application is able to respond within your SLA parameters, such a test would generally be considered a successful component in capacity planning. If the performance metrics recorded during the test are outside of the desirable parameters, a load test may actually become a stress test as you push the system beyond available capacity.
Stress Testing Web App Infrastructure
Identifying the point at which each component in your infrastructure will fail is a critical part of maintaining a scalable web application. Effective stress testing lets you isolate each component through a series of different tests in order to determine the point of failure for that component. Such tests may include:
- Isolating all traffic to one particular geographic region
- Artificially limiting available disk space
- Repeatedly sending one particularly large GET request
- Limiting the maximum number of data connections
- Downloading a large image file
- Repeatedly sending an intense POST that writes heavily to a database
Each test is designed to stress a particular component of the infrastructure to identify the points of failure, the failure rate, and the upper limits of the system capacity. Stress tests can help identify bottlenecks during brief intense loads from things such as viral marketing, international news recognition, and heavy online shopping days such as Black Friday.
The Differences Between Load testing and Stress Testing
Things you might monitor in a stress test:
- CPU Usage
- Free Memory
- Disk IO
- Database reads and writes
- Open database connections
- 3rd party content
If you do not have adequate monitoring systems in place on the server side, Dotcom-Monitor provides a product called MetricsView, which you can install directly on your servers to monitor Windows or Linux performance counters as well as SNMP messages.
Issues with any of these items could be manifested as:
- Slow first packet response
- Long delays between GET/POST requests and responses
- Longer page loads than normal
- Web pages timing out
- Server error codes returned
While these same issues may be initially detected during a load test, the idea behind a load test is generally to simulate expected loads that the system should be able to handle on a regular basis. Sometimes it may be the case where the system briefly experiences slowdowns while resources are being allocated to the increased load, but in most cases the system should be able to recover from the initial allocation and resume normal performance under a load test.
If the load test continues to detect issues, then you need to take a closer look at the system performance counters to determine the root cause of the slowdown or failure. This is when a load test essentially becomes a stress test as the load unexpectedly causes performance issues.
No Credit Card. No Commitment. Pay As You Go.