API testing is a type of software testing wherein an engineer tests not just the functionality but performance, reliability and security for an application. APIs are tested to examine if the application would work the way it is expected to, as APIs are the core of an application’s functionalities.
API testing during development can reveal issues with API, server, other services, network and more, those which one may not discover or solve easily after deployment.
However, testing APIs is difficult. Instead of just verifying an endpoint’s response, one can have integration tests with Postman to examine and validate the responses. Teams these days may also want to automate running these tests as soon as a deployment is done. One approach we can take is to have our integration tests run every time a developer checks in code to the repo.
Adding this layer of Quality check, can ensure that the existing functionalities still work the way they were expected to, with an additional benefit for the developers to validate that their code is doing exactly what it was intended to.
Tools for API test automation in CI
CI refers to continuous integration. Integration of test scripts and a test tool with the continuous build system where the test scripts can be run along with every new deployment or on a regular basis (daily, weekly or fortnightly)
- Postman: Integration tests with Postman.
- Newman: Create a PowerShell file that runs these integration tests via command line.
- Jenkins: Add a Post Build step in Jenkins to execute the PowerShell script whenever a build is initiated.
API Selection
I have implemented this procedure in our Project using the GPS APIs, but for instantiating here, let’s take up the following APIs:
Open Weather Map: Free public APIs.
I chose this as it is a free collection of APIs that anyone can subscribe to and have their own API keys to operate with.
Create Integration Tests
For the first test, let’s take up a simple GET request to get Weather by ID. To interact through the APIs, make sure to use the API key received on subscribing to the OWM services.
Steps to First Integration Test
Make an environment on Postman say, ‘Weather Map’ and define the environment variables in it. [Refer ‘Managing environments’].
Add the Prerequisites in the Pre-Req tab to set up the test.
Collections
Like the above API tests, one can have multiple test scripts for multiple endpoints. And these multiple test scripts can be run in sequence to have an end to end test suite. The way to have a test suite is to keep multiple test scripts in a place holder called as a Collection in Postman.
These collections can then further be executed through the collection runner in the tool.
Collection Runner
A collection runner can be used to have a collection of API endpoints with their test scripts at one place and therefore run them one by one in a sequential manner. The user just needs to run the collection just once with all the required test data, test scripts and for as many iterations one may want. The result of the collection run is a test report, comprehensive enough to monitor the performance of the APIs and also to re-try running the failed test scripts.
For elaborate study on Collection Runners, refer Link.
Though the user interface of Postman’s collection runner is good enough, yet, to integrate the system with Jenkins, we need to run our collections via command line. So, a way to run collections via the command line is through Newman.
Newman
Newman is a Node Package Manager (NPM) package that permits us to run and test collections directly from the command line.
Pre-requisites:
- NodeJS and
- NPM already installed.
Commands to be run on Windows Powershell
- node -v [to verify the version of NodeJs installed]
- npm -v [to verify the version of NPM installed]
- $npm install -g newman [to install Newman]
Once the required installations are done, one needs to have his collections and Environment exported to JSON files in the local system. These files can then be passed as arguments to Newman.
Steps to get the environment and collections on the local system:
- Click on the Download and Export button in Postman.
- Download the collection
- Download the environment
- Open command prompt and raise your privileges. This is important for you to execute the script.
Adding Postman Tests to Jenkins
We first need to export our Postman files (Environment and Collections) and add them to GIT, along with our Powershell script to run the tests through Jenkins build.
“Add the Postman files to the root of the project.”
Telling Jenkins to run Newman
For this we write a script that calls Newman and passes it the Environment and Collection JSON files.
– ‘exit $LASTEXITCODE’: On typing this command, you will see the result of the last command. We do this to make sure that on every loop, the Newman command is successful. If any of the tests fail, we have to stop the script and exit 1. It will result in a failed build on Jenkins.
Adding Script to Jenkins
Steps:
- Login to Jenkins and create a Freestyle Project.
- Start by configuring this project to pull your repo code from Git.
- In the General Tab, go to build section
Running the build and monitoring Results
Try running the project and examine the results.
One can make out successful entrance into the powershell script with the statement ‘Inside Powershell script’ in the Jenkins output.
Conclusion
Improving continuous integration utilizing Postman, Newman and Jenkins adds another layer of quality assurance into our development life cycle. While this is a huge step in automation, we need to emphasize on the fact that our test coverage depends on the quality of our test scripts.
