Load Testing vs. Stress Testing
Load and Stress Testing Compared
A load test, by definition, measures the performance of a system under an expected load.
In contrast, a stress test overloads a system in order to find the breaking point.
Examining the Differences:
Load vs. Stress Testing
A load test is a planned test to perform a specified number of requests to a system in order to test the functionality of the system under specific levels of simultaneous requests. A load test ensures that a web system can handle an expected volume of traffic, and therefore is sometimes referred to as volume testing. The goal of a load test is to prove that a system can handle the expected volume with minimal to acceptable performance degradation. The threshold of acceptable performance degradation must be defined by the testers as some value considered acceptable to the end user so that users will not bounce from the site.
A stress test is a test designed to increase the number of simultaneous requests on a system past a point where performance is degraded, possibly even to the point of complete failure. Where a load test (or API test) will peak out in the number of simultaneous users, a basic stress test will continue to increase load on the system until the resources are overloaded. This pushes the system to a state of potential failure to see how the system handles it and whether the system can perform a graceful recovery.
Within these definitions of a load test and a stress test, we find that they are certainly not completely independent from one another. Often, when running the upper boundaries of a load test, you may effectively end up running a stress test where you push the system past the limits of available resources. At this point you might start to see failures in a load test identical to the failures typically seen when running a stress test.
When to Choose a Load Test or Stress Test
One difference between a load test and stress test is that you may inject pauses into a load test to simulate real user traffic. With a stress test, you may run as many simultaneous users as fast as possible to generate excessive traffic for a stress test.
The goals of a load test are very different than the goals of a stress test. A load test is performed in order to ensure that a website or web application is capable of handling specific numbers of users at once. Load Testing is often used in the process of capacity planning, to ensure that a system can handle growth to specified levels of simultaneous traffic.
A stress test is used to specifically push a system beyond its intended capacity to identify components that begin to slow down, identify bottlenecks in the system, and bring to light possible points of failure.
Commonly Used for Load & Stress Testing:
Establishing Baseline Performance Metrics
Load testing is typically performed as a series of steps where the testing system initiates a quantity of simultaneous users that is known to be supported by the infrastructure. This establishes a baseline set of performance data to reference as the number of simultaneous users is increased throughout the test. This baseline test is sometimes called performance testing. A performance test can help determine several different benchmarks such as average connection speed, average latency, and average time to download a file of a fixed size and more.
Once baseline performance values are known, the number of users is increased to a number that may be realistically expected to visit the site during a sample period. The test will often then run at that static number of users for several minutes in order to verify the stability of the website after the system stabilizes at the new load level.
One difference between establishing a baseline performance metric during a load test and a stress test is that the difference between the baseline and peak performance will help determine if you have the proper systems in place to handle peak load, while during a stress test you are more concerned about the point at which the system becomes stressed and possibly even ceases to work properly.
Load or Stress Testing to Identify Web App Bottlenecks
Web based applications typically run in a browser and when programmed properly, due to their asynchronous nature, may handle many hundreds or thousands of simultaneous users. If you are generating expected load within the capacity of your system, the response times of the application should stay within generated guidelines. If you push the system beyond these limits you move into the realm of stress testing, purposely causing strain on the system to identify the components that fail (this is often performed with open-source tools like JMeter). Therefore, any test performed for the purpose of identifying bottlenecks is typically considered a stress test (which is different from API testing and API monitoring).
Load Testing to Establish Service Level Agreements (SLAs)
Load tests are best performed in a production environment to understand average response times under expected user load. These average response times become the baseline for acceptable SLAs. From here, it is up to you to determine additional thresholds that are considered unacceptable under your SLAs in terms of expected performance for your customers.
Load Testing for Capacity Planning
Generating increased load on a web application can help predict application performance for heavier user load in the future. If the application responds within your SLA parameters, such a test would generally be considered a successful component in capacity planning. If the performance metrics recorded during the test are outside of the desirable parameters, a load test may become a stress test as you push the system beyond its available capacity.
Stress Testing Web App Infrastructure
Identifying the point at which each component in your infrastructure will fail is a critical part of maintaining a scalable web application. Effective stress testing lets you isolate each component through a series of different tests in order to determine the point of failure for that component. Such tests may include:
- Isolating all traffic to a specific geographic region.
- Artificially limiting available disk space.
- Repeatedly sending one particularly large GET request.
- Limiting the maximum number of data connections.
- Downloading a large image file.
- Repeatedly sending an intense POST that writes heavily to a database.
Each test is designed to stress a particular component of the infrastructure to identify the points of failure, the failure rate, and the upper limits of the system capacity. Stress tests can help identify bottlenecks during brief intense loads from things such as viral marketing, international news recognition, and heavy online shopping days such as Black Friday.
The Differences Between Load Testing and Stress Testing
A stress test will typically max out one portion of the system or another which eventually causes slowdowns and then crashes or unresponsiveness. It is important to determine which components in the system will be the first to encounter issues during the test. For this reason, there are several components we recommend you monitor when performing a stress test. It is worth noting that depending on the application, software, or even technology being used in your environment/system, what metrics you measure during a stress test could vary. However, the following list comp
- Response times. Time it takes to receive a response after a request is sent. This is especially important for measuring response times as perceived by users
- Hardware constraints. This includes monitoring CPU usage, RAM, disk I/O. If response times are delayed or slow, these hardware components could be potential culprits.
- Throughput. How much data is sent/received during the test duration based upon your bandwidth levels.
- Database reads and writes. If your application utilizes multiple systems, stress tests can indicate which system, or unit, is the bottleneck.
- Open database connections. Large databases can severely impact performance, slowing response times.
- Third-party content. Web pages and applications rely on many third-party components. Stress testing will show you which ones may impacting your page or application’s performance.
If you do not have adequate monitoring systems in place on the server side, the Dotcom-Monitor platform provides a performance counter monitoring solution for complete end-to-end server performance monitoring. These performance counters are installed directly on your servers to monitor Windows, Linux, or SNMP performance counters. Additionally, there is also a solution for monitoring any custom performance counters from your devices and servers. For more information on performance counter monitoring, visit our Performance Counter Monitoring solutions page.
Issues with any of these items could be manifested as:
- Slow first packet response.
- Long delays between GET/POST requests and responses.
- Longer than normal page load times.
- Web page loading timing out.
- Server error codes returned.
While these same issues may be initially detected during a load test, the idea behind a load test is generally to simulate expected loads that the system should be able to handle on a regular basis. Sometimes it may be the case where the system briefly experiences slowdowns while resources are being allocated to the increased load, but in most cases the system should be able to recover from the initial allocation and resume normal performance under a load test.
If the load test continues to detect issues, then you need to take a closer look at the system performance counters to determine the root cause of the slowdown or failure. This is when a load test essentially becomes a stress test as the load unexpectedly causes performance issues. Contact our team for a demo of the LoadView platform. A performance engineer will walk you through the entire load and stress testing process. From showing how to create scripts for web page or web application scenarios, to configuring and launching the test, they will take you through the whole process and answer any questions you may have along the way.
No Credit Card. No Commitment. Pay As You Go.