Postman Load Testing Tutorial: WDSL, JSON & APIs

APIs are integrated into many of the web services used today, allowing systems to send and communicate various pieces of information back and forth. As such, they are a powerful link between critical services and users. And as with any important part of a system, it is necessary to test them. Using third-party API testing solutions, or Postman, we can do both manual and automated tests (including API monitoring).

API stands for Application Programming Interface. It is a software-to-software interface. With APIs, applications converse with each other with no client intervention. But with thousands of APIs available, Postman comes into picture to create, test, share and manage these APIs making the API usage efficient and less tedious.

This article will help you to understand what Postman is, what Postman Collections are, prominent features and benefites, and how you can set up load tests for Postman collections and its integration with CI/CD pipelines with LoadView.

What is Postman?

Postman is a popular API client tool which makes it easier for development teams to create, share, test, and document APIs. It provides a seamless user experience which helps in hitting API endpoints by quickly creating requests as per the API specification and dissecting the various response parameters like the status code, headers, and the actual response body itself. We can leverage these features for continuous and automated testing. It is used as a companion tool to develop, document and test against an API and provides a user interface to make REST (Representational State Transfer) APIs server requests and shows the response from servers.

Postman is an API development tool used in the software development life cycle:

• API development
• API testing
• API documentation

Postman not only provides us with an environment to work with APIs, but it can also be used to request web pages like a browser. Enter the URL in the address bar and hit send to see the HTML (HyperText Markup Language) response. Postman can also help appending scripts before the user sends the request and after obtaining the response. We can store each request in a collection, which later can be used to run all requests in order.

However, one thing that Postman is not is a performance testing tool. While you will be able to run small, controlled tests that give you API response times, it is not the same as running a test under real world conditions. It does not put the API under any sort of real stress, so it is not a good indicator of how your API will respond when real users are involved. For that, you need a solution like LoadView, which we will talk about in more detail later in this article.

What is API testing?

In API testing we use dedicated tools to send calls to the server hosting the API and retrieve the response. The objective of API testing is to confirm right execution and validating if the API is getting, saving, or updating resources as intended.

If we know that the API is supposed to perform a particular action, like update data, fetch data, delete data, make a change in data, the tests should check the results of those actions. Does it behave the same way as it is supposed to be? What about when it is not supposed to happen? And are the results what you would expect it to be? API testing is typically black-box testing, but to the extent that we have access to the results of the API’s actions beforehand and we know the expected response which needs to be validated.

API Testing: Functional Testing vs. Load Testing

As we briefly mentioned above, functional testing, or black-box testing, is primarily concerned with testing basic functions against a set of specifications and observing whether they work or not. Before APIs can be moved through the development cycle and tested further, it is important that the basic API functionalities are working. If developers move from functional testing to load testing without ensuring basic functions have been properly tested, it will throw off the load tests and results. The results from functional testing are simply observing whether something worked or not.

On the other hand, load testing is like functional testing, but the difference is that load testing sets to test how those functionalities stand up to large numbers of concurrent users, or load. The results of load testing include understanding how those API endpoints respond and perform under load.

Features of Postman

There is seemingly no end to the sheer quantity of features that Postman offers users. From features like the Postman API and workflow controls to built-in monitors for regression testing, the tool has API testing covered. However, at a high-level there are a handful of core features that every API tester will need. Let us take a closer look at these features.

Import Requests

Postman provides an Import feature, where we can paste raw CURL text and convert it into a request formatted with details like Request URL, headers, request body, query parameters and Authorization tokens.

Multiple Protocol and Technology Support

Postman API client feature lets us send REST, SOAP, WSDL and GraphQL requests.

Write Test Cases

We can create test cases which run added APIs and can help in response validation with asserting JSON/XML responses and response code validations.

Collections

Postman helps us to create collections which helps to segregate APIs with folders Structure for different projects. It provides support to run Collection, which will eventually run all the APIs part of the collection.

JSON Schema validation

You can use JSON (JavaScript Object Notation) schema to define the structure of their requests and responses. It helps to clean up and structure JSON content, making it easy to understand and read on Postman.

Use Environment and Dynamic Variables

We can create variables in Postman and based on different environments, the same variables can have different values mapped for the environment selected. This is helpful while we work with the same APIs, but multiple environments in the real world.

Convert API to Code

Postman allows the API request to be converted to code in all the prominent languages like JavaScript, Golang, Java, PHP, Python and many more.

Benefits of Postman

Those who use Postman on a regular basis will be familiar with the benefits below. However, this is not an exhaustive list of capabilities and features of this API testing tool. Let us look at some of the main benefits of Postman.

Automate API Testing

This is the first and most obvious benefit of Postman. The tool lets you automate API testing and integrate the collection with CI/CD pipelines. And as we mentioned above, API testing utilizes either JSON or XML formats, which is independent of the application framework or protocols developers use. No matter what framework you use (Python, JavaScript, Ruby, etc.), Postman supports it. You can automate unit tests, integration tests, regression tests, and mock tests.

Platform and OS Compatible

Postman is available for Mac OS X, Windows, and Linux. The Postman web app is also available and optimized for Chrome, Firefox, Edge, and Safari.

Postman Test Reports

It provides a test report that helps us with tracking data through the request builder and you can generate HTML reports for the test runs.

Integrations

Postman provides support and can be easily integrated with all major developer tools like GitHub, Slack, Dropbox and GitLab.

Pricing

All the major functionalities of Postman are free of cost. You can get all the above benefits for free. There are also paid plans for Postman Pro and Postman Enterprise for small to large enterprises.

Alternatives to Postman

Swagger

Swagger is an open-source API specification, it is like WSDL for REST APIs, where you define the structure of the REST APIs (endpoints URLs, input/output models, authentication schema, etc.). The specification file is based on JSON and the file extension is either JSON or YAML. A few of the key highlights of Swagger are:

• Open-source and free, however it can be hard to learn or find resources for issues.
• Swagger represents APIs within the browser.
• Can automatically generate documentation from building your services.

SoapUI

SoapUI allows users to quickly and easily test REST and SOAP APIs, as the tool was created specifically for testing APIs.  SoapUI helps in testing complicated combinations of web services such as RESTful, SOAP, JSON and AMF.  However, you need to utilize WSDL for testing web services.  Additionally, to ingrate automation, there is a lot of scripting required, and for users who have not had that experience, it can become difficult and time consuming.  A few of the key highlights of SoapUI are:

• Point-and-click functionality helps to makes JSON and XML usability easier.
• Load data from Excel, files, databases, etc.
• Reuse your functional test cases as load tests.
• Integrates with third-party API management platforms, supports REST, SOAP, JMS, and IoT, etc.

Integrate Postman Collection Testing into CI/CD Pipeline

Continuous Integration (CI) is a development practice which requires developers to regularly merge code into a shared repository. It involves the process of automating the build and testing of applications every time a new change gets committed to the source code.  Continuous Development can be explained as a sequence of steps which are required to be performed in sequence to enable application to run on a specialized server and fulfill its use case of serving user requests.

Load Testing Postman Collections using LoadView

Whenever we test a back-end application, testing the functionality is a standard part of the process to ensure that things work as we had intended them to work. Now, that is all well and good, but not for the real world. Your applications will be utilized by many, if not hundreds or thousands of users at the same time, so we also need to carry out performance testing to see how well they hold up to that traffic. We need to benchmark and validate the application with each application change. As we have seen the overview of Postman, its features and how we can set up test strategy for API functional testing, we would need to understand tools which can help to stress test the application.

In cases where we are using Postman to test web API functionality and want to generate a load test from the API call collections we have, we can opt for a specialized load testing tool such as LoadView. LoadView provides real browser-based load testing for APIs, as well as web applications (external and internal), and websites and servers.

Test Setup

We will take a step-by-step look on how to set up a web application load test for a Postman collection.

LoadView Postman API Testing:  Step 1

Before we even start, we would need to export the Postman Collection. We can choose the option to Get Public link and can easily import it to LoadView (Note: Make sure the Postman Collection uses local variables only since environment variables are not stored in the collection).

LoadView Postman API Testing: Step 2

Open the LoadView platform and select Create New load test.

LoadView Postman API Testing: Step 3

Here you will see the various load testing types that are available within LoadView – web apps, websites and APIs. For our use case, we will select the Postman Collection option to start API tests.

LoadView Postman API Testing: Step 4

A new window will open and here is where we will need to import the exported Postman collection and click Create Device.

LoadView Postman API Testing: Step 5

Once we have successfully created a device, we would see the Test Scenario screen, where we can set Load Type, which would differ based on the goal of our test.

• Load Based Curve.  This is to execute load tests with a known number of users and raise traffic after set warmup time.
• Goal Based Curve.  This test setup is used when we are looking for desired transactions per second for our specific API and want to scale up to the desired concurrent users.
• Dynamic based Curve.  This set provides you to choose dynamic values in number of users, maximum users and test duration, and can be changed, in real-time, during the test.

LoadView Postman API Testing: Step 6

Once we have created a test scenario with a list of all APIs, we can now execute load and stress tests for our APIs. After successful completion of the load test run, you will be provided with reports, dashboards, and metrics that show how our API and systems performed under load.

Postman Collection Load test integration with Jenkins

As we have looked at how we can integrate and run Postman collection load tests using LoadView, we can add these tests as part of CI/CD to get regular feedback and performance results. LoadView also provides integration with Jenkins which helps stress test websites, web applications, and APIs with concurrent connections in real browsers from a fully managed cloud. Any new updates or additional features can also be load tested with the LoadView plugin for Jenkins.

Let us take a look at the steps used for the previously created Postman collection test scenario in LoadView and its integration with Jenkins.

Step 1.  To integrate the tests with Jenkins, we would be using the same Postman test scenario created above.

Step 2.  Create a unique UID. This will be used as a security token to integrate into Jenkins.  Copy the Integration UID.

Step 3.  Test Scenario ID for our Postman collection can be found on the scenario setup page or on the test History page (Test Manager > the test actions menu > History).

LoadView Plugin in Jenkins

To configure the LoadView plugin, follow these steps:

1. Log into your Jenkins account.
2. In Jenkins, go to Credentials > Add Credentials > LoadView Security Token (UID). Configure and validate the credentials.

• Kind.  LoadView Security Token (UID).
• Scope.  Select Global.
• ID. Leave the default or specify a unique ID.
• Description.  Specify a unique description to distinguish the credentials.
• UID.  Paste the load testing web API UID from your LoadView account. Once the UID has been added, click Validate UID to make sure Jenkins can access the LoadView API.

3. Next, select a job and click Configure.

4. Go to Build > Post-build Actions > Add post-build action > LoadView-Run load test scenario. Specify LoadView Stress Test settings for the build:

• Credentials. Select the API key using the description.
• Scenario ID. Paste the test scenario ID you want to use in the build. Copy the ID from the scenario page in LoadView.
• Error Threshold.  Specify an acceptable percentage of sessions with errors that occurred while testing (failed to access the target resource, no keywords/image found, etc.). If the percentage of error sessions is greater than the specified value, the build will be marked as a failure in Jenkins.
• Average Time.  Specify a limit for an average response time measured during test execution. If the limit is reached, the build will be marked as a failure in Jenkins.

5. Click Save.

View LoadView Test Results

Finally, we have successfully integrated our Postman collection with Jenkins using LoadView as a platform. While our stress test is running, the status is displayed in the Jenkins Console Output in real-time mode.

Monitoring Postman Collection Test

LoadView is one part of the many solutions within the Dotcom-Monitor platform. Dotcom-Monitor also provides easy monitoring tools for our Postman collections. If we have a collection of integration tests used for internal API testing with Postman, and we also want to set up tests to run from outside the local network, we can upload the Postman tests to Dotcom-Monitor. Using the Dotcom-Monitor Postman Collection task we can create automated monitoring tests with alerts on errors. We can specify monitoring locations, a monitoring scheduler, filters, and set up reporting based on monitoring results.

The collection of HTTP requests added will be repeatedly executed with the defined frequency to perform the sequence of calls to the target web API. With that we will have the assertions that were specified for the requests in Postman which will help to verify API responses. If any condition set by assertions is not met the task will be marked as failed.

Load Testing Your API with Postman:  Final Thoughts

The development community has widely varying opinions about testing APIs. We have seen how Postman helps us altogether in creating, maintaining, and testing out APIs with such ease. We also looked at a step-by-step guide on how we can simulate concurrent users load testing Postman collection using LoadView. Lastly, we have looked at how we can automate this step using Jenkins integration with LoadView. The tools required to stress out these applications have been simplified lately and it provides immense help testing out complex systems that are dependent on internal and external applications. It helps us to spot the bottlenecks and issues more clearly.

At LoadView, our goal is to be your expert partner for all things load testing, at every stage of development and any level of complexity. We understand load testing can be a chore, so we work tirelessly to make the process easy so you and your development team can integrate the load testing process into your existing DevOps practices and devote the lion’s share of your time and resources to what matters – namely, improving your websites and web applications for your users. By taking the headaches out of load testing, LoadView helps to improve your entire development process and ultimately your users’ experience.

Our team of experts is available to support you and your team as you develop and enhance your load testing strategy, and you’ll find our platform is intuitive and easy to use even for non-technical users. We offer real browser testing, point and click scripting, and comprehensive, readable reports that take the guesswork out of load testing and allow for effective collaboration across developer functions. We also work to stay in front of the load testing industry and deliver our users new and refined tools to meet changing demands of the constantly evolving digital landscape. LoadView is your complete load testing platform, now and for the future.

Start load testing your APIs today with LoadView.  Sign up for the free trial and receive $20 in load testing credits.  Or sign up for one of our private demos with one of our performance engineers. Our engineer will walk you through the full LoadView platform, as well as show you how to integrate LoadView into Postman collections and automating the steps with Jenkins.