Load Testing vs. Stress Testing
Load and Stress Testing Compared
A load test, by definition, measures the performance of a system under an expected load.
In contrast, a stress test overloads a system to find the breaking point.
Examining the Differences:
Load vs. Stress Testing
A load test is a planned test to perform a specified number of requests to a system to test the functionality of the system under specific levels of simultaneous requests. A load test ensures that a web system can handle an expected volume of traffic, and therefore is sometimes referred to as volume testing. The goal of a load test is to prove that a system can handle the expected volume with minimal to acceptable performance degradation. The threshold of acceptable performance degradation must be defined by the testers as some value considered acceptable to the end user so that users will not bounce from the site.
A stress test is a test designed to increase the number of simultaneous requests on a system past a point where performance is degraded, even to the point of complete failure. Where a load test (or API test) will peak out in the number of simultaneous users, a basic stress test will continue to increase load on the system until the resources are overloaded. This pushes the system to a state of potential failure to see how the system handles it and whether the system can perform a graceful recovery.
Within these definitions of a load test and a stress test, we find that they are certainly not completely independent from one another. Often, when running the upper boundaries of a load test, you may effectively end up running a stress test where you push the system past the limits of available resources. At this point you might start to see failures in a load test identical to the failures typically seen when running a stress test.
One difference between a load test and stress test is that you may inject pauses into a load test to simulate real user traffic. With a stress test, you may run as many simultaneous users as fast as possible to generate excessive traffic for a stress test.
The goals of a load test are quite different compared to the goals of a stress test. A load test is performed to ensure that a website, web application, or API is capable of handling specific numbers of users at once. Load testing is often used in the process of capacity planning, to ensure that a system can handle growth to specified levels of simultaneous traffic.
A stress test is used to specifically push a system beyond its intended capacity to identify components that begin to slow down, identify bottlenecks in the system, and bring to light potential bottlenecks or points of failure.
Load testing is typically performed as a series of steps where the testing system initiates a quantity of simultaneous users that is known to be supported by the infrastructure. This establishes a baseline set of performance data to reference as the number of simultaneous users is increased throughout the test. Performance test can help determine several different baselines, such as average connection speed, average latency, and average time to download a file of a fixed size and more.
Once baseline performance values are known, the number of users is increased to a number that may be realistically expected to visit the site during a sample period. The test will often then run at that static number of users for several minutes to verify the stability of the website after the system stabilizes at the new load level.
It is important to note that the terms baseline testing and benchmark testing are often used interchangeably, however, there are differences between these two terms. Baseline testing is carried out to ensure that the performance of a site or application does not decrease over time. For example, during a baseline test, performance metrics are recorded so that when that application or site is updated in future, engineers can test and compare the new performance metrics with the previous metrics. These baseline tests would also include any new code, software, hardware, and network changes. The goal is to deliver a consistent application or site, which in turn ensures a positive experience for users.
Benchmark testing is the practice of comparing application performance against specific pre-defined industry or organizational standards and requirements. Like baseline testing, benchmark testing includes measuring and recording performance of hardware, software, and network conditions. Benchmark testing helps to measure the quality of service for an organization’s own requirements or against other organizations. These metrics help to create SLAs (Service Level Agreements) for organizations and provide a guaranteed service level for users or customers. Read more about baseline and benchmark testing.
One difference between establishing a baseline performance metric during a load test and a stress test is that the difference between the baseline and peak performance will help determine if you have the proper systems in place to handle peak load, while during a stress test you are more concerned about the point at which the system becomes stressed, or even ceases to work properly.
Load or Stress Testing to Identify Web App Bottlenecks
Web based applications typically run in a browser and when programmed properly, due to their asynchronous nature, may handle many hundreds or thousands of simultaneous users. If you are generating expected load within the capacity of your system, the response times of the application should stay within generated guidelines. If you push the system beyond these limits you move into the realm of stress testing, purposely causing strain on the system to identify the components that fail (this is often performed with open-source tools like JMeter). Therefore, any test performed for the purpose of identifying bottlenecks is typically considered a stress test (which is different from API testing and API monitoring).
Load Testing to Establish Service Level Agreements (SLAs)
Load tests are best performed in a production environment to understand average response times under expected user load. These average response times become the baseline for acceptable SLAs. From here, it is up to you to determine additional thresholds that are considered unacceptable under your SLAs in terms of expected performance for your customers.
Load Testing for Capacity Planning
Generating increased load on a web application can help predict application performance for heavier user load in the future. If the application responds within your SLA parameters, such a test would be considered a successful component in capacity planning. If the performance metrics recorded during the test are outside of the desirable parameters, a load test may become a stress test as you push the system beyond its available capacity.
Stress Testing Web App Infrastructure
Identifying the point at which each component in your infrastructure will fail is a critical part of maintaining a scalable web application. Effective stress testing lets you isolate each component through a series of different tests to determine the point of failure for that component. Such tests may include:
- Isolating all traffic to a specific geographic region.
- Artificially limiting available disk space.
- Repeatedly sending one particularly large GET request.
- Limiting the maximum number of data connections.
- Downloading a large image file.
- Repeatedly sending an intense POST that writes heavily to a database.
Each test is designed to stress a particular component of the infrastructure to identify the points of failure, the failure rate, and the upper limits of the system capacity. Stress tests can help identify bottlenecks during brief intense loads from things such as viral marketing, international news recognition, and heavy online shopping days such as Black Friday.
A stress test will typically max out one portion of the system or another which eventually causes slowdowns and then crashes or unresponsiveness. It is important to determine which components in the system will be the first to encounter issues during the test. For this reason, there are several components we recommend you monitor when performing a stress test. It is worth noting that depending on the application, software, or even technology being used in your environment/system, what metrics you measure during a stress test could vary.
- Response times. Time it takes to receive a response after a request is sent. This is especially important for measuring response times as perceived by users
- Hardware constraints. This includes monitoring CPU usage, RAM, disk I/O. If response times are delayed or slow, these hardware components could be potential culprits.
- Throughput. How much data is sent/received during the test duration based upon your bandwidth levels.
- Database reads and writes. If your application utilizes multiple systems, stress tests can indicate which system, or unit, is the bottleneck.
- Open database connections. Large databases can severely impact performance, slowing response times.
- Third-party content. Web pages and applications rely on many third-party components. Stress testing will show you which ones may impact your page or application’s performance.
If you do not have adequate monitoring systems in place on the server side, the Dotcom-Monitor platform provides a performance counter monitoring solution for complete end-to-end server performance monitoring. These performance counters are installed directly on your servers to monitor Windows, Linux, or SNMP (Simple Network Management Protocol) performance counters. Additionally, there is also a solution for monitoring any custom performance counters from your devices and servers. For more information on performance counter monitoring, visit our Performance Counter Monitoring solutions page.
Issues with any of these items could be manifested as:
- Slow first packet response.
- Substantial delays between GET/POST requests and responses.
- Longer than normal page load times.
- Web page loading timing out.
- Server error codes returned.
While these same issues may be initially detected during a load test, the idea behind a load test is to simulate expected loads that the system should be able to handle on a regular basis. Sometimes it may be the case where the system briefly experiences slowdowns while resources are being allocated to the increased load, but in most cases the system should be able to recover from the initial allocation and resume normal performance under a load test.
If the load test continues to detect issues, then you need to take a closer look at the system performance counters to determine the root cause of the slowdown or failure. This is when a load test becomes a stress test as the load unexpectedly causes performance issues. Want to learn more? Contact our team for a demo of the LoadView platform. A performance engineer will walk you through the entire load and stress testing process. From showing how to create scripts for web page or web application scenarios, to configuring and launching the test, they will take you through the entire process and answer any questions you may have along the way.
Our team at LoadView is available to support you and your development team to get the most out of your load and stress testing process and budget. By understanding when to run load and stress tests respectively, backed by our easy-to-use platform and helpful experts, you’ll be able to integrate an intelligent testing process into your DevOps strategy and quickly deliver results for users of your websites and web applications.
As with most things in the fast-changing digital landscape, it’s important to leverage expert help. At LoadView, our entire focus is on the development of our industry-leading platform for load and stress testing. We help our clients focus on what matters when load testing: accurate, actionable results that can be turned into insights that drive meaningful changes in the development of websites and applications and supporting infrastructure.
Without regular load and stress testing, your websites and applications are at risk of performance degradation or serious downtime. Even if you’re next to certain your websites are crash proof, proper testing will help you and your development team make improvements that can give even more peace of mind and a better experience for your users around the world. In the online world, speed matters. When you use LoadView as your go-to load and stress testing platform, you’ll be able to deliver the best possible experience for your users and ensure uptime. Sign up for a free LoadView trial today.
No Credit Card. No Commitment. Pay As You Go.