Soak Testing Websites and Web Applications

It’s good practice to test your website for performance before and after deploying it to the production environment. Apart from that, you should also do performance testing in production if you expect an increase in demand due to any event. Website load testing is a widely used performance testing for determining how your website and underlying systems will behave with a large number of users. There are a few approaches for load testing, and in this article, we will discuss the two of them, soak testing and peak testing.

Soak Testing Approach for Load Testing

Soak testing, also known as endurance testing, measures websites, and web application performance under a huge number of users for an extended period of time. The main purpose of soak testing is to understand the performance and time trade-off beyond the system expectations. It is known that if a website which can endure the high number of users for a limited duration, let’s say one hour, is exposed to a huge load continuously for 5 hours, 10 hours, even longer, the website can crash or behave in an unexpected manner. Soak testing is performed to predict and determine abnormal behavior of the website under such conditions.

Example Scenario for Soak Testing

A simple example of soak testing would be a gaming website. When a user plays any game on a website, lots of server files are created for the session, some of which get cleaned up after the session. Such a website might perform well under high load for a shorter period of time, but if the load continues to be in longer sessions, memory leaks can happen, or the memory can be exhausted completely. This will cause the server to crash, and all the users will be affected.

Advantages and Benefits of Soak Testing

Memory allocation and utilization visibility to help planning for additional resources.
Useful to determine auto scaling strategy with a prioritized set of resources.
Determining database resources and the number of connections for uninterrupted performance.
Response time and load exposure time trade-off insights to plan for additional measurements of garbage collection and memory allocation.
It discovers potential issues that the standard load testing approach fails to detect.
Validate infrastructure demands while doing IT budgeting.

Disadvantages and Challenges of Soak Testing

Soak testing is time-consuming and requires additional memory resources for a longer duration.
It is hard to estimate how long the test should run, you may run it for 10 hours and give it an ‘ok,’ but your website might break after 11 hours, which you can never know.
If your website is live, you must avoid running soak testing in a live environment. Running in a different environment requires extra resources which could potentially be the same as your production environment, adding up the cost.

Peak Testing Approach for Load Testing

Peak testing is done for determining peak load, which is the maximum number of concurrent users on your website at any given time. The idea behind the peak testing is to simulate user load, which you expect during peak hours. It is used to determine CPU usage, database requests per second, response delays under expected user surge during peak hours. Unlike soak testing, peak testing does not require running it for prolonged periods of time. It only requires generating maximum traffic that you speculate may come during peak hours, like weekend, night, trading hours, etc.

Example Scenario for Peak Testing

Let’s say you have a trading website. On such a platform, high traffic would be during normal trading hours. In such case, you might have 10,000 users who might come concurrently on your website during trading hours. Normally you would do load testing with average load, for example, 1,000 concurrent users. But for peak testing instead of generating average load, you want to generate the peak load that you expect during those hours.

Advantages and Benefits of Soak Testing

Helpful in determining system benchmarks.
Identifying system bottleneck that can bog down your website during peak hours.
Reducing the risk of downtime during peak hours.
Identification of scalability scope for the entire infrastructure.
Minimized cost of failure.

Disadvantages and Challenges of Soak Testing

Peak testing can fail to identify issues related to endurance testing.
High cost to generate huge amounts of virtual users.
One bottleneck can hinder another bottleneck and require multiple re-runs to accurately determine the maximum peak load.

A Holistic Approach: Soak Testing and Peak Testing

While both soak testing and peak testing have their advantages and disadvantages in different scenarios, if you use a strategy that incorporates both, then you can discover more information with increased accuracy about your website. For instance, peak testing is not done for prolonged periods of time, but if you include peak load in your soak testing, you can determine how long your website can withhold with the maximum number of concurrent users. For our trading example scenario, It would be recommended to do soak testing for trading hours with peak load.

Cloud-based or On-premises Testing

Soak testing and peak testing require specialized hardware and a significant amount of resources to generate appropriate load for a significant amount of duration. Not every organization or website owner who wants to have a stable website with a high number of users can afford this dedicated infrastructure. Let’s look at when you should opt for either one.

Use cloud-based testing when:

You don’t want to purchase all the hardware and software required.
You want to test your website from multiple locations.
If you want to create the most realistic conditions such as multiple locations, multiple browsers, multiple devices with cost-effectiveness.

Use on-premises testing when:

You have enough funds for testing infrastructure and maintenance teams.
You do not need to test your website from multiple locations.
You have a frequent daily requirement for load testing.

LoadView for Soak Testing

LoadView offers a cloud-based load testing environment with flexible test designs to cover almost any scenario for website and web application performance testing. It uses real-browsers and multiple devices from different locations to generate the most realistic condition for any test scenario. You can design your user scripts for the Load test with no coding experience using its EveryStep Web Recorder. Soak tests in the LoadView platform can be performed using its Load Step Curve feature, which allows you to specify a set number of concurrent users for a specific time duration. This can be used for soak testing and peak testing individually and for the combination of both testings. Apart from that, you can also adjust your load in real-time to adjust situations to better understand the performance under fluctuating conditions.