Performance Testing with Postman: Is It Worth It?
What is Postman?
01.
Enter your request details
02.
The API server
03.
Postman receives the response and displays it in the interface
What type of performance testing do you need?
Find out on a free consultation with our experts.
Ways to use Postman for load testing
We have found three basic scenarios of using Postman to test performance.
- Running embedded tests, for example, measuring the response time.
- Trying to create the load with Collection Runner and Newman.
- Writing special scripts for parallel execution of requests.
Way 1: Measure response time in Postman.
- Add a new collection in the sidebar.
- Add a new GET request with variables.
- Go to the Tests tab.
- Choose Response time is less than 200ms from the list of snippets.
- Under Collections, select Run.
Yes, it is that easy! The problem is… such tests have nothing to do with performance testing. In fact, all requests in Postman are sent one by one, whilst the basic idea of load tests is to check how the system copes with simultaneous requests from a large number of users.
Way 2: Create multiple requests in Postman Collection Runner
You might want to argue that Collection Runner in Postman was designed specifically to create multiple requests! Indeed, developers have added this tool so that users could send multiple requests to the API. It still has nothing to do with performance testing, though.
Collection Runner is designed to automate tests. It allows you to specify a sequence of API requests, transfer data between them, use various environments, and perform other tasks. Seems like a great solution, but the keyword here is still “sequence”, and each operation will be run one after another.
And even if you add thousands of iterations to your requests, they will all be run in a sequence. In other words, your API will only handle one request per a time period and will not experience any load.
There are more “exotic” ways to abuse Collection Runner and make it carry out requests simultaneously, but their efficiency is highly questionable.
Way 3: Force Postman to launch several collections simultaneously using Newman
Some enthusiasts go even further and try to create scripts that allow them to run multiple collections simultaneously. To run one of them you need to leave the handy interface and use Postman’s CLI Newman — a tool to manage the Collection Runner via the console. It is designed to integrate functional API testing into the CI/CD pipeline, but maybe we should give it a try.
The scripts themselves are easy to find on GitHub. As the description says, they are intended to run multiple collections in parallel, which should mimic simultaneous requests from users. This seems closer to performance testing, as the API receives many requests simultaneously and will experience some load.
The problem is, in this case you will have zero control over the process. For example, processing a high number of requests could lead to a CPU or RAM overloading and test failure, while you will not receive any information about the runs at all. Even if you get lucky and everything goes well, the results of such a test would not show you a real picture of API performance, because it is not scenario-based. In other words, real people usually don’t send thousands of requests in a moment, and Postman does not allow you, for instance, to gradually increase the load.
Which method really works?
Unfortunately, none of the above. Postman works perfectly for functional and integration testing, but creating the load for APIs is simply something out of its scope. If you need to test your API performance, it’s highly recommended to use tools specifically designed for the task, such as the PFLB platform.
It has a user-friendly interface and does not require any coding skills either. But most importantly, it will provide you with an objective result, help you make conclusions about your API performance, and fix the problems that need fixing.
