Performance Testing with Postman: Is It Worth It?

Postman is a quite popular tool for checking APIs’ work and running functional and integration tests easily. However, many developers also try to use Postman for performance testing. Some of them simply measure the response time with available functions, while others go as far as writing scripts from scratch to try and trick the system into simulating the load. In this blog post, we will share our opinion of the foundations of these methods and discuss if they are worth your time.

What is Postman?

Postman is one of the most famous tools for working with different API types, such as REST, gRPC, GraphQL, and even SOAP. Millions of developers design, create, and test application programming interfaces with Postman. One of the tool’s greatest advantages is that you can send requests to APIs and endpoints and retrieve data from the source without going to the console or writing a single line of code.

01.

Enter your request details

02.

The API server

03.

Postman receives the response and displays it in the interface

This feature helped Postman get the reputation of a user-friendly tool to debug and test APIs “in one click”. Indeed, it is very convenient and useful and does well with functional and integration tests. However, in addition to these, many try to run load, performance, and stress tests with Postman, although the software is not designed for such tasks. Let’s try to understand how enthusiasts manage to do load testing with Postman, and what results they get.

What Type of Performance Testing Do You Need?

Find out on a free consultation with our experts.

Ways to use Postman for load testing

We have found three basic scenarios of using Postman to test performance.

Running embedded tests, for example, measuring the response time.
Trying to create the load with Collection Runner and Newman.
Writing special scripts for parallel execution of requests.

The choice usually depends on the level of developers’ expertise and the time they are ready to spare for the task. It’s just about time to find out if performance testing of APIs in Postman does work.

Way 1: Measure response time in Postman.

The most popular operation in Postman is testing individual API queries, which is quite logical since the tool was designed primarily for functional testing. When they speak of performance testing in Postman, they usually mean one particular test – response time measurement. It is rather easy to run:

Add a new collection in the sidebar.
Add a new GET request with variables.
Go to the Tests tab.
Choose Response time is less than 200ms from the list of snippets.
Under Collections, select Run.

Yes, it is that easy! The problem is… such tests have nothing to do with performance testing. In fact, all requests in Postman are sent one by one, whilst the basic idea of load tests is to check how the system copes with simultaneous requests from a large number of users.

Way 2: Create multiple requests in Postman Collection Runner

You might want to argue that Collection Runner in Postman was designed specifically to create multiple requests! Indeed, developers have added this tool so that users could send multiple requests to the API. It still has nothing to do with performance testing, though.
Collection Runner is designed to automate tests. It allows you to specify a sequence of API requests, transfer data between them, use various environments, and perform other tasks. Seems like a great solution, but the keyword here is still “sequence”, and each operation will be run one after another.

And even if you add thousands of iterations to your requests, they will all be run in a sequence. In other words, your API will only handle one request per a time period and will not experience any load.

There are more “exotic” ways to abuse Collection Runner and make it carry out requests simultaneously, but their efficiency is highly questionable.

Way 3: Force Postman to launch several collections simultaneously using Newman

Some enthusiasts go even further and try to create scripts that allow them to run multiple collections simultaneously. To run one of them you need to leave the handy interface and use Postman’s CLI Newman — a tool to manage the Collection Runner via the console. It is designed to integrate functional API testing into the CI/CD pipeline, but maybe we should give it a try.

The scripts themselves are easy to find on GitHub. As the description says, they are intended to run multiple collections in parallel, which should mimic simultaneous requests from users. This seems closer to performance testing, as the API receives many requests simultaneously and will experience some load.

The problem is, in this case you will have zero control over the process. For example, processing a high number of requests could lead to a CPU or RAM overloading and test failure, while you will not receive any information about the runs at all. Even if you get lucky and everything goes well, the results of such a test would not show you a real picture of API performance, because it is not scenario-based. In other words, real people usually don’t send thousands of requests in a moment, and Postman does not allow you, for instance, to gradually increase the load.

Which method really works?

Unfortunately, none of the above. Postman works perfectly for functional and integration testing, but creating the load for APIs is simply something out of its scope. If you need to test your API performance, it’s highly recommended to use tools specifically designed for the task, such as the PFLB platform.

It has a user-friendly interface and does not require any coding skills either. But most importantly, it will provide you with an objective result, help you make conclusions about your API performance, and fix the problems that need fixing.