Alternatives to Telerik Test Studio:
LoadView by Dotcom-Monitor: Powerful, On-demand Load Testing from the Cloud
On-demand, web-based load and stress testing for all your websites, applications, and APIs.
See performance under load with Android, Chrome, Internet Explorer, and iOS
Easy to Use Scripting Tool
Choose from 13+ locations from around the world
Pay as you go pricing with no annual renewal fees
Gain insights into performance and identify system bottlenecks
Forget the hassle of having to provision hardware or installing additional software. Ensure your infrastructure scales to meet the demands of your users with LoadView.
|Features||Telerik Test Studio||Other Cloud Providers||LoadView|
|No pre-packaged minimum on virtual users|
|No Annual Renewal Fee|
|Utilize Cloud-based Load Injectors|
|No separate platform fee for performance testing|
|Pay as You Go Pricing|
Websites and web applications are becoming more dynamic and complex every day, with user expectations becoming increasingly demanding. In fact, studies have shown that if your website doesn’t load within just a few seconds, users are likely to abandon it completely, and likely never return. Organizations need to utilize a performance testing platform that allows them to quickly and easily test the performance of their websites and applications, without the hassle of having to manage additional infrastructure or cloud providers.
The LoadView and Test Studio platforms both allow engineers and DevOps teams to load test their web applications, websites, and APIs to ensure performance and a consistent user experience. And while there are some feature similarities between LoadView and Test Studio, there are some key differences between the two platforms that we will set out to explain in the article below.
LoadView vs. Test Studio: A Quick Overview
Before digging into the details of both platforms, let’s take a quick look at some of the main differences. LoadView was created to remove time consuming aspects of load testing, so engineers and developers can focus on creating and executing their tests. Tasks such as setting up load generators and creating global agents are managed completely by LoadView, so users you don’t have to worry about spending time on provisioning any hardware, software, and additional infrastructure or worrying about having to setup your own cloud account. And the LoadView platform is cloud-based, so even someone who has never had experience running a load test can setup and execute a load test within minutes. All of these efficiencies open up more time for developers to fully dedicate their effort and time to doing what they do best: designing, executing, and analyzing their tests.
On the other hand, Telerik Test Studio was designed more for executing functional UI tests for web, mobile, and desktop applications that integrates into an organization’s CI/CD process (Jenkins, Team Foundation Server, Jira, etc.). Their entry level solution doesn’t include load testing, so users have the option to upgrade to a license that includes performance and load testing. Test Studio supports popular technologies, such as Angular, React, WPF (Windows Presentation Foundation), MVC, AJAX, RESTful, etc. Additionally, users have the option to use scripts built within Telerik’s own framework for building web and mobile apps, called Kendo UI.
One of the major drawbacks to Test Studio is that it is not a web-based solution. Test Studio must be downloaded onto a local machine and users are given a set number of virtual users to pull from based upon your license. LoadView is cloud-based and uses the AWS and Google Cloud Platforms, so you can spin up as many virtual users as you need for your specific testing requirements.
Test Studio System Requirements
- .NET Framework – 4.5.2
- Visual Studio – 2017, 2015, 2013, and 2012 Professional or higher
- C# / Visual Basic .NET
Supported Operating Systems
- Windows 10, 8.1, 8, 7
- Windows Server 2016, 2012, 2008
- Internet Explorer 11
- Microsoft Edge
- Mozilla Firefox (latest official version)
- Google Chrome (latest official version)
- Safari for Windows 5.1.7
With LoadView, all you need is an Internet connection and a browser. You don’t have to worry about upgrading based on your testing scenario. In the case of Test Studio, if you ever need a larger virtual user pool to pull from, you’re limited to only few add-on options and the costs can get into the thousands of dollars, depending on the number of virtual users you require. Additionally, there is an annual renewal fee to ensure you receive and upgrade to the latest version of Test Studio.
LoadView: Load Testing for Today’s Websites, Applications, and Web Services
Many of the free, open-source load testing solutions on the market today can only run protocol-based user simulations, which is not enough for today’s demanding websites and applications. You can stress the server at the protocol level, but a significant part of end-to-end response time is neglected. This is important when it comes to testing web applications that comprise of Rich Internet Applications (RIA), such as AJAX, HTML5, Flash, and Java. Understanding how the client-side performance is impacted is vital to ensure a consistent user experience.
The LoadView platform gives users options when it comes time to test user simulations – protocol-based and real browser-based tests. Here are some key points to consider when deciding on a method that best suits your specific testing needs:
HTTP/S-Based Load Simulation
- No client-side rendering
- Programming skills required for scripting
- Low-level analysis required
- Low impact on load injection machines
Real Browser-Based Load Simulation
- Real user simulation
- Easy to customize scripts
- Script replay
- High impact on load injection machines
LoadView, along with the EveryStep Web Recorder, can record every step of a web transaction and replay the script in a real browser. The EveryStep Web Recorder is a free, web-based automation scripting tool and requires no additional plugins or downloads. Users can record multi-step test scripts for complex user actions, such as button clicks, menu selections, form submission, shopping carts, text inputs, and more. These scripts can then be uploaded into the LoadView platform for load testing.
The EveryStep Web Recorder allow users to record in over 40 desktop/mobile browsers and devices (Chrome, Internet Explorer, iOS, iPhone, iPad, Android, Samsung, and more). The EveryStep Web Recorder supports virtually anything rendered in a browser, regardless of the technologies used, and supports the following set of popular languages/technologies:
|Web API (REST)||SOAP||WebSockets|
To give users more customization and flexibility, LoadView also provides several options when it comes to defining the load type for your test.
- Load Step Curve
Generates loads with a pre-determined number of concurrent users for specified time durations. Useful when determining if your website can handle a spike in traffic.
- Goal-based Curve
This test is useful when you have already identified the limiting level of your required throughput and need to ensure that your performance goal is met. Great for validating SLA or non-functional requirements.
- Dynamic Adjustable Curve
Allows users to adjust load in real-time. This gives users insight into how their site or application changes as load levels fluctuate.
Global Testing and Detailed Performance Reporting
Once you’ve determined the load type and your virtual users, one of the final steps is to decide where you’re going to test from. LoadView gives users the option to distribute user load between geographic locations. For example, if you know that a certain percentage of your customers come from a specific geographic location, you can select those specific zones to test from. LoadView utilizes cloud servers (AWS and Google) located in 13 locations worldwide to generate user loads. Servers are located throughout the United States, Canada, APAC, South America, and Europe.
Once the load test has finished, users can drill into the specific components within provided performance reports, including region-specific results, in addition to performance metrics essential to your organization’s capacity planning.
Telerik Test Studio: Performance Testing
The Test Studio platform is a bit different in the fact that, even though load testing is typically defined as a type of performance testing, performance testing and load testing are two separate testing types within the platform. For Test Studio users, that means performance and load testing begins where functional testing leaves off, and because Test Studio is primarily used as a functional testing tool, one of the benefits for users is that they can reuse functional tests for performance and load tests. This cuts down the maintenance time for developers and gives them more reliability and mileage out of each test.
Since staging/QA environments are typically separate from production/live environments, it is recommended to run a performance test in a no-load environment to get a baseline to start from. You will then want to run a performance test in an average or peak load environment, so you get a good feeling of where that trend is and where it should be. Performance testing gives users detail on every step of each scenario, such as a login process and uses profiler/counters to get performance metrics, like memory, CPU, disk, etc.
Performance tests within Test Studio are not focused on the functional nature or failures, they serve to give you an idea of the server and client-side timing out of all the steps. If there is a failure, you can go back to the functional test to troubleshoot/fix any errors. Performance testing in a peak load environment is about locating the bottlenecks within a particular application or scenario, based on a specific load volume, so users can focus development efforts to make applications more efficient.
Within Test Studio there is a Detail section that gives users visibility into a waterfall chart on the performance of each step and what needs to be improved, if anything. There is also an overlay of all of the performance counters (up to hundreds of them) that provide metrics on processing time, server session detail, errors, etc. There is a lot of information that can be displayed, which can be overwhelming to review at one time, so Test Studio users do have the option to turn off all the performance counters or turn on the ones that apply to your specific needs.
Telerik Test Studio: Load Tests
Load testing within the Test Studio platform is, like mentioned earlier, a different type of test. When users open a project in Test Studio, they are given multiple test types to choose from, including Web, Load, WPF (Windows Presentation Foundation), and Manual test types. But the same concept of reusing functional test scripts for load testing applies here. There are a number of different components and configurations of the load test.
What Test Studio does is capture HTTP traffic generated by certain user scenarios (called user profiles within Test Studio) and build together a composite of what typical traffic would look like. For example, users can run multiple user profiles, which could be anything from a login process or updating records to browsing a page or navigating through the shopping cart process. Within Test Studio, users have four option from where the HTTP traffic comes from:
- Imported from an existing load test.
- Imported from a Fiddler trace.
- Manually capture new traffic from a selected web browser.
- Capture HTTP traffic generated by executing an existing Telerik functional web test in a selected browser.
Similar to scripting with the EveryStep Web Recorder and LoadView, the idea is to create, or build, scenarios that match what your users are doing. For Test Studio users, those processes, or user profiles, can be part of existing functional web tests. Test Studio users have the option to select from multiple browsers. Test Studio supports Internet Explorer, Firefox, Chrome, and Safari, as well as Android and iOS mobile devices. However, one important note about using mobile devices is that the device is only needed to capture the traffic, not executing the load test. Once the traffic is captured, the test executes as a headless browser, not a real browser like LoadView would.
Additionally, similar to LoadView, Test Studio allows users to input delays, or think time. This can be set manually so each of the virtual users acts more like a real user would. Each virtual user will vary slightly, giving your test more of a real-world scenario. You can also set pre-defined unique user info or credentials coming from a data source, such as usernames and passwords.
Once the test is finished, Test Studio allows users to see the traffic that has been captured. The first item that comes up are Dynamic Targets. Dynamic Targets are the session ID’s that were detected by Test Studio. Test Studio doesn’t specifically target all session ID’s, since there are some out there that are more advanced, but the rule is to just turn them all on. During a load test, you typically don’t want to execute any 3rd party elements, like ads, social media, etc. Filtering those out allows you decide on the items you don’t need at the domain level. You could also go through the list and disable them one by one. Additionally, Windows performance counters can be included in the load test results. That is one feature that is different in their load testing versus performance testing.
Test Studio: Geographic Testing
Test Studio includes 100 virtual users with each of their Ultimate licenses. So, if your team buys two licenses, you have 200 virtual users to use and they can be allocated between different machines, however it’s not necessary to distribute them from multiple machines, but only if you want to split the pool of users across different scheduling servers.
Within Test Studio, there is one central scheduling server that talks to many machines, automatically. During a load test, the scheduling server will automatically allocate users across the agents. You give the scheduling server “x” amount and it will automatically distribute across the different machines that are running different environments, browsers, etc. Or your teams may have more than one test lab, meaning multiple scheduling servers and environments.
Test Studio Load can also perform geographic diversity testing by installing and selecting Execution Servers in different locations. These locations can simulate the locations of different users or user populations and the results can be compared against the two. The process to define all of your execution severs and configure the settings can be time-consuming, which is why LoadView makes it easy by allowing you to just select which locations you’d like to test from. With Test Studio, there is a list of considerations that need to be met before configuring the execution server.
LoadView Free Trial and Pricing
LoadView has subscription-based plans, starting at $199 per month and we offer a 30-day free trial with $20 in load testing credits. Alternatively, you can also select from a pay-as-you-go plan. Pricing is calculated and displayed before you begin new test, so you know exactly what you’re paying for.
If your specific needs require more than what is listed above, please contact us for Enterprise-level plans.
LoadView Professional Services
The goal of Dotcom-Monitor Professional Services is to have you running performance tests, analyzing your data, and adjusting your system without our engagement. Our engineers are also available to perform the entire testing exercise for you, offering consultation to optimize your code and infrastructure. Our methodology is tested, proven, and offered in unique à la carte fashion. You pick and choose which elements you need assistance with, and our performance engineers will coach you through them.
Test Studio Free Trial
Must be downloaded to your local machine. The 30-day free trial contains the following:
- Unlimited tech support
- Support and learning resources
- Intuitive cross-browser test recorder
- GUI web and desktop testing
- Performance and Load testing (20 Virtual Users included)
- API testing
- Visual Studio plugin
- Exploratory testing
- Manual testing support
- HP Quality Center, MS TFS Source Control and Bug Tracking support, Git integration
Test Studio Pricing
Telerik offers a couple of different perpetual licenses, however, in order to take advantage of load testing, you must purchase the $3,499 Test Studio Ultimate perpetual license. Included are the following features:
- Automated web and desktop testing
- Visual Studio plugin
- Manual testing
- Exploratory testing
- Functional UI testing
- Test scheduling and execution
- Parallel test execution on multiple machines
- Continuous integration support
- Mobile testing
- API testing
- Performance testing
- Load testing
- Priority Support (24 hours guaranteed initial response time)
- 100 virtual users for load testing
- For additional virtual users, separate add-on licenses must be purchased.
$899 – 100 virtual users
$2,999 – 1,000 virtual users
$4,999 – 10,000 virtual users
*All Test Studio licenses include the storage and scheduling and server capabilities. You can have multiple runtime environments with different full licensing. The full licenses (Test Studio Functional/Test Studio Ultimate) build and maintain the tests.
Both the LoadView and Test Studio are powerful performance testing platforms. However, when it comes to ease of use, cost-effectiveness, and real browser-based load and stress testing, LoadView is the clear choice here. The ability to create scripts that simulate real user behavior by just pointing and clicking and utilizing the cloud to easily generate virtual users from multiple points around the globe, makes it the go-to platform. With Test Studio, there is a steep learning curve. There are so many additional steps, configurations, and requirements that need to be executed before running any tests. It’s a complex product that can be used for many times of testing, which is great for organizations that have large budgets, but if you’re looking for platform that is easy to use, pay-as-you go pricing, and real-world test results, look no further than LoadView.
Try LoadView from Dotcom-Monitor
and get $20 in Free Testing Credits!