Clicky

 

Learn about Load Testing

What Makes an Outstanding Load Testing Platform

 



Successful companies understand that the speed of their business applications becomes more and more a top priority because frustrated customers move quickly away to competitor’s web sites. Every lost customer has an impact on financial revenue. Therefore, businesses have started to improve reliability of their applications and spend more money on load testing and application monitoring. The former helps to eliminate hotspots pre-deployment into production. The latter is mainly used to identify outages or slowdowns and isolate the root cause of issues during production stages.

In this piece, we will shine a light on load testing platforms, give some differentiators and describe why you should use LoadView in your next performance engineering assignment.

Different Types of Load Testing Solutions

Almost 30 years back in time the first web pioneers first created load simulation platforms. At this time, web pages were simple and content was mainly static. With the rise of technology many services are nowadays dynamic and constantly changing. Competition is high, and businesses try to hold existing or gain new customers with even better services. One opportunity to improve service quality is to provide responsive and reliable applications.

In recent years new load and performance testing tool solutions have appeared in this growing market. The pioneers such as JMeter or LoadRunner were deployed inside company’s local network on their machines. With the rise of cloud computing some of them extended their service to SaaS or on-demand-based load testing platforms.

On-Premise

There are open source and commercial on-premise load testing solutions, which are deployed on a customer’s local infrastructure. While open source tools are free of charge, the vendors of commercial solutions charge initial license fees and deliver the software. The customer deploys updates on his local testing servers by themselves. If some load testing script or execution challenges occurs a support specialist can be involved to investigate and fix the identified issue.

On-Demand or SaaS

Maintaining a local load-testing infrastructure can be challenging. Therefore, successful tool provider switched to a cloud based offering which avoids the pain of operating local load testing suites. The benefit is that there are no maintenance tasks or fees involved and the client pays for the required service.

How Does Load Simulation Work?

Manual tests are designed to detect functional issues such as a feature that is not working as intended, however, these types of tests are not feasible for testing how your new application behaves under production like load conditions. Performance testing helps to validate the reliability of your new system under actual or future growth load volumes. In contrary to manual testing, load testing is a highly technical task and often requires experienced specialists.

The basic steps of a load test setting are

  1. Calculate the load pattern
  2. Identify the relevant user interactions
  3. Automate the testing scripts
  4. Configure the test scenario such as regular and peak load
  5. Assign the load injection machines
  6. Execute the tests
  7. Analyze the results
  8. Fix defects and repeat the test until hotspots have been fixed

All on-premise or on-demand performance test platforms support the mentioned steps to some extent. Open source solutions provide limited user simulation techniques. Some new on-demand or SaaS based providers try to convince new customers with extremely high free user simulation opportunities. You can waste your money with both and it is crucial to evaluate those carefully before you make your decision. In the next few chapters we will give you some guidance around how to select the best load testing suite.

The Importance of Self-Monitoring During Load Generation

Many popup load testing platforms offer simulation of thousands virtual users for free but what can you expect from their proposals? Do you think that they will provide their powerful, cloud based load injection machines for free? Obviously, this is not the case and they just promise something which won’t work at all.

Based on our experience with hundreds of performance testing projects, an accurate and reliable load testing farm can do three main things:

  1. Monitoring of the load on load injection machines
  2. Inform the user about actual health of load generators during a test run
  3. Never share load generators for multiple tests at the same point in time

Aggressive competitors in the performance testing business often share load injectors across many tests and simulate a high number of virtual users on a single machine. The consequence is that response times are far too high and the expected load pattern can’t be achieved at all. From a customer’s perspective a few thousands concurrent users for free is very attractive but if you look carefully and think about this offer you will come to realize that there are some pitfalls involved.

The first being that you can’t trust response times captured on an overloaded load test injector. Simulated users on such machines will fight for CPU and memory resources. Those servers will start paging and the processor queue will increase and impact user experience metrics negatively.

Secondly, due to the high load on the injection machine the requests will arrive delayed at your application under test, which leads to a wrong load pattern. Your system being monitored will eventually easily handle the low number of requests.

Finally, you will start tuning the system in a wrong way because the incorrect response times and the low number of requests will point you into the wrong direction. After your tuning and re-run of those invalid load tests with totally overloaded load injection machines you will still experience the same issues.

To sum up this section, it’s highly recommended to use a load testing solution such as LoadView, which monitors its load agents permanently during a test execution, provides health monitoring charts while the test is executed, and never shares load generators with other tests.

What Makes a Good Load-Testing Platform?

It’s safe to assume that many companies are not happy with their load-testing suite because they are too expensive, usability is pure, and maintenance efforts are high and re-use of automated testing scripts is extremely limited.

Based on our experience an outstanding performance-testing platform fulfills these key requirements:

  • Ease of use: Create a script and run a load test within several hours
  • Technology support: Support capture and replay of the top three browser, mobile devices, web2.0 techniques, Java, Citrix, SAP and .Net based applications
  • Realistic user simulation: Provide protocol, headless and full browser based user simulation techniques
  • Reusability: Use automated user interactions for performance test and application monitoring on production stages
  • Support community: Knowledge base and professional support which provides guidance to fix automation or tooling issues
  • Maintainability: Close to zero tool maintenance effort
  • Costs: Pay by use is the preferred charging model
  • Self-monitoring: Distribute load across injection machines and built-in features to avoid that overloaded machines falsifies load test results

LoadView – An Even Better Load Testing Suite

Why should businesses setup and maintain their own performance-testing suite? Obviously, this is a waste of time and money because the effort to install the load injection software exceeds often the time required for test automation, execution and result analysis. LoadView is a fully SaaS-based service hosted in the cloud. There is no need to deploy your private load generation machine. You can create your account and immediately start with test automation and execution.

The majority of performances testing solutions are user-unfriendly. It will take you ages to understand how they work and without experienced engineers you might be lost. LoadView is everything but complicated. It’s fully web-based UI and intuitive design lets you create test scripts and execute tests within minutes. No coding skills are required to automate complex user interactions.

LoadView addresses the problem of unrealistic load patters with its flexible user simulation techniques, from protocol to real browser based. Furthermore, LoadView calibrates your automated scripts, distributes those across load generation machines and automatically avoids situations where an overloaded injection machine falsifies your response times.

Execution of load and performance tests during development and QA stages is highly recommended and gives you confidence that your new application can handle production user and transaction volumes within expected boundaries. Nevertheless, user, content and data volumes will change over time and can lead to serious reliability issues. Therefore, performance is more a journey than a destination and you should permanently monitor end-to-end response times and availability on production stages. In a best case scenario, you can reuse your load testing assets for application monitoring at production. LoadView provides this flexibility and comes up with a closed loop testing approach enabling you to reuse testing scripts for production monitoring.

Load Testing Platform Evaluation Sheet

There are hundreds of load testing solutions on the market and companies have often no idea, which should be selected, and what decision criteria should be used. Therefore, we’ve created a decision matrix.

How to use this matrix

  1. Answer this form for each load testing solution on your short list
  2. Select the appropriate rating for each criteria
  • ++ means that it completely fulfills the criteria
  • + means that this is a good fit
  • – means that it somewhat meets the criteria
  • — means that it doesn’t meet the criteria in any way
  1. Calculate the score
  • Each x in ++ column is a 3
  • Each x in + column is a 2
  • Each x in – column is a 1
  • Each x in – column is a 0

The matrix below shows the rating of LoadView. Its score of 23 (24 would be the max value) is extremely high and demonstrates that LoadView is a perfect match for forward thinking and cost aware companies.

Criteria Rating of LoadView
 ++  +  –  —
Ease of use x
Technology support x
User simulation x
Reusability x
Support community x
Maintainability x
Self monitoring x
Costs x
21 2 0
Total Score 23

Don’t waste your money—replace your underperforming load testing solution with LoadView. Get started today and get a load test running in 5 minutes or less!


Login