fbpx

APIs are integrated into many of the web services used today, allowing systems to send and communicate various pieces of information back and forth. As such, they are are a powerful links between critical services. And as with any important part of a system, it is necessary to test them. Using third-party API testing solutions, or Postman, we can do both manual and automated tests.  API stands for Application Programming Interface. It is a software to software interface. With APIs, applications converse with each other with no client intervention. But with thousands of APIs available, Postman comes into picture to create, test, share and manage these APIs making the API usage efficient and less tedious.

This article will help to understand Postman, collections, its prominent features, and how we can set up load tests for Postman collections and its integration with CI/CD pipelines with LoadView.

 

What is Postman?

Postman is a popular API client tool which makes it easier for development teams to create, share, test and document APIs. It provides a seamless user experience which helps in hitting API endpoints by quickly creating requests as per the API specification and dissecting the various response parameters like the status code, headers, and the actual response body itself. We can leverage these features for continuous and automated testing. It is used as a companion tool to develop, document and test against an API and provides a user interface to make REST APIs server requests and shows the response from servers.

Postman is an API development tool used in the software development life cycle:

  • API development
  • API testing
  • API documentation

Postman not only provides us with an environment to work with APIs, It can also be used to request web pages  like a browser. Enter the URL in the address bar and hit send to see the HTML response. Postman can also help appending scripts before the user sends the request and after obtaining the response. We can store each request in a collection, which later can be used to run all requests in order.

 

What is API testing?

In API testing we use dedicated tools to send calls to the server hosting the API and retrieve the response. The objective of API testing is to confirm right execution and validating if the API is getting, saving or updating resources as intended.

If we know that the API is supposed to perform a particular action, like update data, fetch data, delete data, make a change in data, the tests should check the results of those actions. Does it behave the same way as it’s supposed to be? What about when it’s not supposed to happen? And are the results what you would expect it to be? API testing is typically black-box testing, but to the extent that we have access to the results of the API’s actions and we beforehand know the expected response which needs to be validated.

 

Features of Postman

 

Import Requests

Postman provides an Import feature, where we can paste raw CURL text and convert it into a request formatted with details like Request URL, headers, request body, query parameters and Authorization tokens.

 

Multiple Protocol and Technology Support

Postman API client feature lets us send REST, SOAP, WSDL and GraphQL requests.

 

Write Test Cases

We can create test cases which run added APIs and can help in response validation with asserting JSON/XML responses and response code validations.

 

Collections

Postman helps us to create collections which helps to segregate APIs with folders Structure for different projects. It provides support to run Collection, which will eventually run all the APIs part of the collection.

 

JSON Schema validation

We can use JSON Schema to define the structure of their requests and responses. It helps to beautify and structure JSON making it easy to understand and read on Postman.

 

Use Environment and Dynamic Variables

We can create variables in Postman and based on different environments, the same variables can have different values mapped for the environment selected. This is helpful while we work with the same APIs, but multiple environments in the real world.

 

Convert API to Code

Postman allows the API request to be converted to code in all the prominent languages like JavaScript, Golang, Java, PHP, Python and many more.

 

Benefits of Postman

 

Automate Testing

Postman let us automate API testing and integrate the collection with CI/CD pipelines. We can automate Unit tests, integration tests, regression tests and mock tests as part of this tests

 

Platform and OS Compatible

Postman is available on Mac, Windows, Linux and Chrome and supports all major operating systems.

 

Postman Test Reports

It provides a test report that helps us with tracking data through the request builder and we can generate HTML reports for the test runs.

 

Integrations

Postman provides support and can be easily integrated with all major developer tools like Github, Slack, Dropbox and Gitlab.

 

Pricing

All the major functionalities of Postman are free of cost. We can get all of the above benefits for free. There are also paid plans for Postman Pro and Postman Enterprise for small to large enterprises.

 

Alternatives to Postman

 

Swagger

Swagger is an open-source API specification, it is like WSDL for REST APIs, where you define the structure of the REST APIs (endpoints URLs, input/output models, authentication schema, etc.). The specification file is based on JSON and the file extension is either JSON or YAML. A few of the key highlights of Swagger are:

  • Open-source and free, however it can be hard to learn or find resources for issues.
  • SwaggerUI represents APIs within the browser
  • Can automatically generate documentation from building your services

 

SoapUI

SoapUI allows users to quickly and easily test REST and SOAP APIs, as the tool was created specifically for testing APIs.  SoapUI helps in testing complicated combinations of web services such as RESTful, SOAP, JSON and AMF.  However, you need to utilize WSDL for testing web services.  Additionally, to ingrate automation, there is a lot of scripting required, and for users who haven’t that experience, it can become difficult and time consuming.  A few of the key highlights of SoapUI are:

  • Point-and-click functionality helps to makes JSON and XML usability easier.
  • Load data from Excel, files, databases, etc.
  • Reuse your functional test cases as load tests.
  • Integrates with third-party API management platforms, supports REST, SOAP, JMS, and IoT, etc.

 

 

Integrate Postman Collection Testing into CI/CD Pipeline

Continuous Integration (CI) is a development practice which requires developers to regularly merge code into a shared repository. It involves the process of automating the build and testing of applications every time a new change gets committed to the source code.  Continuous Development can be explained as a sequence of steps which are required to be performed in sequence to enable application to run on a specialized server and fulfill its use case of serving user requests.

We can integrate Postman collections APIs to run inside our Continuous Integration/Continuous Development environments using Newman, Newman is a command line tool that allows us to run a postman collection on our server or on a local environment. To integrate we would need a Collection UID and Postman API key. We can run postman collection with a single command as simple as mentioned below:

newman run https://api.getpostman.com/collections/{{collection_uid}}?apikey={{postman-api-key-here}}

 

Load Testing Postman Collections using LoadView

Whenever we test a back-end application, testing the functionality is a standard part of the process to ensure that things work as we had intended them to work. That’s all well and good, but not for the real world. Your applications will be utilized by many, if not hundreds or thousands of users at the same time, so we also need to carry out performance testing to see how well they hold up to that traffic.  We need to benchmark and validate the application with each application change. As we have seen the overview of Postman, its features and how we can set up test strategy for API functional testing, we would need to understand tools which can help to stress test the application.

In cases where we are using Postman to test Web API functionality, and want to generate a load test from the API call collections we have, we can opt for a specialized load testing tool such as LoadView. LoadView provides real browser-based load testing for APIs, as well as web applications (external and internal), and websites and servers.

 

Test Setup

We ‘ll take a look, step-by-step, on how to setup a web-app load test for a Postman collection.

LoadView Postman API Testing:  Step 1

Before we even start, we would need to export the Postman Collection. We can choose the option to Get Public link and can easily import it to LoadView (Note: Make sure the Postman Collection uses local variables only since environment variables are not stored in the collection).

Postman Collection Export

 

LoadView Postman API Testing: Step 2

Open the LoadView platform and select Create New load test.

 

LoadView Postman API Testing: Step 3

Here you will see the various load testing types that are available within LoadView – web apps, websites and APIs. For our use case, we will select the Postman Collection option to start API tests.

Load testing type Postman

 

LoadView Postman API Testing: Step 4

A new window will open and here is where we’ll need to import the exported Postman collection and click Create Device.

Postman Collection Request

LoadView Postman API Testing: Step 5

Once we have successfully created a device, we would see the Test Scenario screen, where we can set Load Type, which would differ based on the goal of our test.

  • Load Based Curve.  This is to execute load tests with a known number of users and raise traffic after set warm up time.
  • Goal Based Curve.  This test setup is used when we are looking for desired transactions per second for our specific API and want to scale to the desired rate slowly.
  • Dynamic based Curve.  This set provides you to choose dynamic values in number of users, maximum users and test duration, and can be changed, in real-time, during the test.

 

Postman Collection Load Types

 

LoadView Postman API Testing: Step 6

Once we have created a test scenario with a list of all APIs, we can now execute load and stress tests for our APIs. After successful completion of the load test run, you will be provided with reports, dashboards, and metrics that show how our API and systems performed under load.

 

 

Postman Collection Load test integration with Jenkins

As we have looked at how we can integrate and run Postman collection load tests using LoadView, we can add these tests as part of CI/CD to get regular feedback and performance results. LoadView also provides integration with Jenkins which helps stress test websites, web applications, and APIs with concurrent connections in real browsers from a fully managed cloud. Any new updates or additional features can also be load tested with the LoadView plugin for Jenkins.

Let’s take a look at the steps used for the previously created Postman collection test scenario in LoadView and its integration with Jenkins.

Step 1.  To integrate the tests with Jenkins, we would be using the same Postman test scenario created above.

Step 2.  Create a unique UID. This is will be used as security token to integrate into Jenkins.  Copy the Integration UID.

Postman Jenkins Integration UID

 

Step 3.  Test Scenario ID for our Postman collection can be found on the scenario setup page or on the test’s History page (Test Manager > the test’s actions menu > History).

Postman Collection Test History

 

LoadView Plugin in Jenkins

To configure the LoadView plugin, follow these steps:

  1. Log into your Jenkins account.
  2. In Jenkins, go to Credentials > Add Credentials > LoadView Security Token (UID). Configure and validate the credentials.
  • Kind.  LoadView Security Token (UID).
  • Scope.  Select Global.
  • ID.  Leave the default or specify a unique ID.
  • Description.  Specify a unique description to distinguish the credentials.
  • UID.  Paste the load testing Web API UID from your LoadView account. Once the UID has been added, click Validate UID to make sure Jenkins can access the LoadView API.
  1. Next, select a job and click Configure.

Jenkins LoadView Security Token

 

  1. Go to Build > Post-build Actions > Add post-build action > LoadView-Run load test scenario. Specify LoadView Stress Test settings for the build:
  • Credentials. Select the API key using the description.
  • Scenario ID.  Paste the test scenario ID you want to use in the build. Copy the ID from the scenario page in LoadView.
  • Error Threshold.  Specify an acceptable percentage of sessions with errors that occurred while testing (failed to access the target resource, no keywords/image found, etc.). If the percentage of error sessions is greater than the specified value, the build will be marked as a failure in Jenkins.
  • Average Time.  Specify a limit for an average response time measured during test execution. If the limit is reached, the build will be marked as a failure in Jenkins.
  1. Click Save.

 

View LoadView Test Results

Finally, we have successfully integrated our Postman collection with Jenkins using LoadView as a platform. While our stress test is running, the status is displayed in the Jenkins Console Output in real-time mode.

Postman Jenkins Console Output

 

Postman Jenkins LoadView Test Performance Results

 

Monitoring Postman Collection Test

LoadView is part of the Dotcom-Monitor solution. Dotcom-Monitor also provides easy monitoring tools for our Postman collections. If we have a collection of integration tests used for internal API testing with Postman, and we also want to set up tests to run from outside the local network, we can upload the Postman tests to Dotcom-Monitor. Using the Dotcom-Monitor Postman Collection task we can create automated monitoring tests with alerts on errors. We can specify monitoring locations, a monitoring scheduler, filters, and set up reporting based on monitoring results.

The collection of HTTP requests added will be repeatedly executed with the defined frequency to perform the sequence of calls to the target Web API. With that we will have the assertions that were specified for the requests in Postman which will help to verify API responses. If any condition set by assertions is not met the task will be marked as failed.

 

Load Testing Your API with Postman:  Final Thoughts

The development community has widely varying opinions about testing APIs.  We’ve seen how Postman helps us altogether in creating, maintaining, and testing out APIs with such ease. We also looked at a step-by-step guide on how we can simulate concurrent users load testing Postman collection using LoadView. Later we have looked at how we can automate this step using Jenkins integration with LoadView. The tools required to stress out these applications have been simplified lately and it provides immense help testing out complex systems that are dependent on internal and external applications. It helps us to spot the bottlenecks and issues more clearly.

Start load testing your APIs today with LoadView.  Sign up for the free trial and receive $20 in load testing credits.  Or sign up for one of our private demos with one of our performance engineers. We’ll walk you through the full LoadView platform, as well as show you how to integrate LoadView into Postman collections and automating the steps with Jenkins.