They have employed PFLB to test the system and to tune its performance.
to examine the possibilities of increasing the load from the app and the site, from the couriers and restaurant counters, because the client needed a system performance evaluation ahead of the regional marketing campaign.
to investigate the possibilities of distributing the system load through the year.
The system contains several applications that take care of the following business processes: accepting customer orders through the web-interface and mobile apps; accepting restaurant branch orders from call-center employees and shift managers; accounting the quality and the speed of executing the order.
During testing most of the load was coming from the site or mobile app users and the cashiers that take orders. The secondary load sources emulated the system workflow and influenced the business processes: the tracker (it controls the order execution), TV-board in the pizzerias (it requests information from the system) and the information screens.
We’ve written test cases containing script and emulator run schedule, as well as scripts that emulated user actions, retrieved the amount of current operations, their intensity and the number of virtual users performing them, recorded the test duration.
We’ve used the hardware resource monitoring that was employed in the client’s system that consisted of grafana + influxDB solution + telegraf(windows)/ exporter(*nix).
Based on the test results we’ve analyzed the system load criteria for different load levels, including overcoming the maximal performance. As the analysis result we’ve listed the critical areas requiring monitoring on production and the factors that decrease performance: the lack of hardware or system resources, the software architecture limits. We’ve also selected the most resource-greedy and longest operations to optimize.
The first difficulty was the startup workflow. The company hasn’t had the routine, where the load testing would have fit in. Much of the time was lost on configuring the load testing environment.
There was also no recent service specification. We had to learn everything on the flow.
Because of the frequent releases (2-3 in a day) we had to update the scripts all the time, because every update could change the system processes.