Choosing an Appropriate Load Testing Platform
Often times, organizations do not reach the expected ROI from their load testing investments because usability is poor, maintenance efforts are high, and re-use of automated testing scripts is extremely limited. In this post, this page will examine essential characteristics of load testing platforms and provide some guidance how to make your selection.
Over the past decade, a long list of open-source or cloud-based load testing solution providers have appeared in this growing market. The need for reliable applications has reached an all-timehigh and organizations now spend more on performance engineering than ever before. In some cases, the reason for excessive spending is simply the use of wrong tools. An outstanding load testing platform should fulfill some, or all, of the essential requirements mentioned below.
Ease of Use
All steps involved in load testing such as script creation, test execution, and results analysis should be lightweight and intuitive. As a rule of thumb, you should be able to create a script and run a load test within a few hours.
A good load testing suite provides capture and replay support for the most frequently used devices and technologies such Firefox, Chrome, Internet Explorer, mobile devices, Web 2.0 techniques, and APIs.
Realistic User Simulation
Limited user simulation techniques can lead to inaccurate test results. With the rise of browser-based applications, the focus is more on real browser-based or headless user simulation. In rare cases protocol-based applications may be required. Providing adequate user simulation techniques is one of the most important criteria, because incorrectly simulating user interactions will falsify your test and make your results useless.
Load testing goes hand in hand with application monitoring on production. It’s a waste of money if you implement testing and monitoring scripts twice. Outstanding load testing platforms will protect your investment and enable reuse of those for production monitoring.
Load testing of user interactions can be tricky and often requires help of experienced developers or specialists. It’s essential to get access to a knowledge base and professional support specialists because they can guide you through scripting or tooling issues.
Rollout of updates on load testing platforms is often seen as an overhead and brings additional risks. Therefore, choose a load testing solution which protects you from such maintenance pitfalls.
Initial license costs are a pain for companies and, more often than not, they decide to go with open-source tools. Preferably, the initial investment on load testing tools should be low. Pay by use is the preferred charging model. Costs are an important factor, but also consider that when open-source load testing tools are hosted on your infrastructure and you have to deal with updates or limited simulation techniques.
It’s highly recommended to use a load testing solution that monitors its load agents continuously during a test execution, provides health monitoring charts while the test is executed, and never shares load generators with other tests.
Load Testing Platform Evaluation Matrix
There are hundreds of load testing solutions on the market. Often, companies have no idea of the breadth of options available,, which specific solution to choose from, and what decision criteria should be used. To aid in choosing a testing solution, please see the matrix listed below:
How to use this matrix:
A.) Apply this matrix to each load testing solution on your short list
B.) Add the appropriate rating for each criterion
4 = Completely fulfills the criteria
3 = Mostly fulfills the criteria
2 = Does not fulfill the criteria
1 = Barely fulfills the criteria
C.) Calculate the score
Don’t waste time and money with an outdated load testing platform. Use the evaluation matrix, compare your current load testing tool with a few commercial and open-source solutions, and consider using a modern load testing suite to protect your investment or business online.