Postman is a quite popular tool for checking APIs’ work and running functional and integration tests easily. However, many developers also try to use Postman for API performance testing. Some of them simply measure the response time with available functions, while others go as far as writing scripts from scratch to try and trick the system into simulating the load. In this blog post, we will share our opinion of the foundations of these methods and discuss if they are worth your time.
What Is Postman?
Postman is one of the most famous tools for working with different API types, such as REST, gRPC, GraphQL, and even SOAP. Millions of developers design, create, and test application programming interfaces with Postman. One of the tool’s greatest advantages is that you can send requests to APIs and endpoints and retrieve data from the source without going to the console or writing a single line of code.
This feature helped Postman get the reputation of a user-friendly tool to debug and test APIs “in one click”. Indeed, it is very convenient and useful and does well with functional and integration tests. However, in addition to these, many try to run load, performance, and stress tests with Postman, although the software is not designed for such tasks. Let’s try to understand how enthusiasts manage to do load testing with Postman, and what results they get.
Ways to Use Postman for Load Testing
We have found three basic scenarios of using Postman to test performance.
The choice usually depends on the level of developers’ expertise and the time they are ready to spare for the task. It’s just about time to find out if API load testing in Postman does work.
Way 1: Measure response time in Postman.
The most popular operation in Postman is testing individual API queries, which is quite logical since the tool was designed primarily for functional testing. When they speak of performance testing in Postman, they usually mean one particular test – response time measurement. It is rather easy to run:
Yes, it is that easy! The problem is… such tests have nothing to do with performance testing. In fact, all requests in Postman are sent one by one, whilst the basic idea of load tests is to check how the system copes with simultaneous requests from a large number of users.
Way 2: Create multiple requests in Postman Collection Runner
You might want to argue that Collection Runner in Postman was designed specifically to create multiple requests! Indeed, developers have added this tool so that users could send multiple requests to the API. It still has nothing to do with performance testing, though.
Collection Runner is designed to automate tests. It allows you to specify a sequence of API requests, transfer data between them, use various environments, and perform other tasks. Seems like a great solution, but the keyword here is still “sequence”, and each operation will be run one after another.
And even if you add thousands of iterations to your requests, they will all be run in a sequence. In other words, your API will only handle one request per a time period and will not experience any load.
There are more “exotic” ways to abuse Collection Runner and make it carry out requests simultaneously, but their efficiency is highly questionable.
Way 3: Force Postman to launch several collections simultaneously using Newman
Some enthusiasts go even further and try to create scripts that allow them to run multiple collections simultaneously. To run one of them you need to leave the handy interface and use Postman’s CLI Newman — a tool to manage the Collection Runner via the console. It is designed to integrate functional API testing into the CI/CD pipeline, but maybe we should give it a try.
The scripts themselves are easy to find on GitHub. As the description says, they are intended to run multiple collections in parallel, which should mimic simultaneous requests from users. This seems closer to performance testing, as the API receives many requests simultaneously and will experience some load.
The problem is, in this case you will have zero control over the process. For example, processing a high number of requests could lead to a CPU or RAM overloading and test failure, while you will not receive any information about the runs at all. Even if you get lucky and everything goes well, the results of such a test would not show you a real picture of API performance, because it is not scenario-based. In other words, real people usually don’t send thousands of requests in a moment, and Postman does not allow you, for instance, to gradually increase the load.
Which method really works?
Unfortunately, none of the above. Postman works perfectly for functional and integration testing, but creating the load for APIs is simply something out of its scope. If you need to test your API performance, it’s highly recommended to use a specialized API load testing tool specifically designed for the task, such as PFLB.
It has a user-friendly interface and does not require any coding skills either. But most importantly, it will provide you with an objective result, help you make conclusions about your API performance, and fix the problems that need fixing.
Related insights in blog articles
SRE Roles and Responsibilities: Key Insights Every Engineer Should Know
Site Reliability Engineers (SREs) are crucial for maintaining the reliability and efficiency of software systems. They work at the intersection of development and operations to solve performance issues and ensure system scalability. This article will detail the SRE roles and responsibilities, offering vital insights into their duties and required skills. Key Takeaways Understanding Site Reliability […]
Understanding Error Budgets: What Is Error Budget and How to Use It
An error budget defines the allowable downtime or errors for a system within a specific period, balancing innovation and reliability. In this article, you’ll learn what is error budget, how it’s calculated, and why it’s essential for maintaining system performance and user satisfaction. Key Takeaways Understanding Error Budgets: What Is Error Budget and How to […]
Mastering Reliability: The 4 Golden Signals SRE Metrics
Introduction to Site Reliability Engineering Site Reliability Engineering is a modern IT approach designed to ensure that software systems are both highly reliable and scalable. By leveraging data and automation, SRE helps manage the complexity of distributed systems and accelerates software delivery. A key aspect of SRE is monitoring, which provides real-time insights into both […]
Reliability vs Availability: Key Differences
Defining Reliability and Availability What is Reliability? Reliability refers to the probability that a system will consistently perform as expected, delivering correct output over a set period of time. In the world of Site Reliability Engineering (SRE), reliability is a core metric that drives everything we do. It’s not just about whether a service works […]
Be the first one to know
We’ll send you a monthly e-mail with all the useful insights that we will have found and analyzed
People love to read
Explore the most popular articles we’ve written so far
- Benefits of Performance Testing for Businesses Sep 4, 2024
- Android vs iOS App Performance Testing: What’s the Difference? Dec 9, 2022
- How to Save Money on Performance Testing? Dec 5, 2022
- Cloud-based Application Testing: Features & Types Apr 15, 2020