How We Automated Performance Testing for an ATM Control System

It is often impossible to provide high-quality performance testing for a complex system without automation. Today, we would like to share our experience of carrying out such a task for the largest government-owned bank, which has a chain of over 190 thousand ATMs. For our client, performance automation helped organize all systems in a strong and effective network.

As everyone knows, banks have different types of self-service machines: some dispense cash, others print tickets for the virtual queue, some perform money transfers, and still others transfer funds to mobile phones. The functionality differs, but one aspect remains the same: all devices require maintenance.

Do all queue management machines and ATMs have up-to-date software? Are all devices checked regularly? In the event of a failure, how quickly do the maintenance teams rectify the fault? Automated control systems help manage all of these considerations. One such system was involved in the project described below.

But it was not the only system involved. In the modern world, everything is interconnected, so when a bank has a new application, customers are informed about it on ATMs, in mobile applications, and even on entrance screens.

The ATM control system communicates with several neighboring systems, including separate web and mobile applications for physical and legal persons, SmartVista software, and an authentication system.

One of the services shown above is responsible for display communication in bank offices and for receiving and displaying tickets. Another service evaluates the importance of the customers when they visit a bank office.

We also had to conduct Jenkins load testing of these systems while working on the project. We will share how we completed these testing tasks.

Task

The customer assigned us the task of creating performance testing automation for the ATM control system. This way, the client could speed up critical business processes, reducing the amount of time and resources expended.

Solution

Automation software testing was used for the ATM control system to speed up test preparation and create the Pipeline to use Jenkins in performance testing.

To speed up and simplify the performance testing in the ATM testing process, we used pipelines. Currently, the pipeline job in the Jenkins system defines the sequence of actions that will be executed after the Start Test button is clicked. The sequence of actions looks like this:

Current script versions are in a GitLab repository connected to Jenkins via Git Integration, similar to WebHooks. After changes are made to the repository, Jenkins checks to see which files were changed after the last script update in MicroFocus Performance Center. If an update is required, a job is automatically invoked that collects scripts and required files in a predefined folder.

A test is selected based on the predefined job parameters in Jenkins, such as searching for the maximum, minimum confirmation, and reliability testing.

The disadvantage of this approach is that the Jenkins load testing profile in the maximum search must be set manually. In other words, the Pipeline could be further optimized by automatically determining the maximum. It would allow the maximum confirmation test to be created or modified based on the results obtained.

Agents in Influx collect information about the systems’ work during the test execution. The results of MicroFocus Performance Center are also stored there. The information is used to create Grafana charts.

We would like to cover the result collection programs in a little more detail. One such program accumulates Grafana diagrams selected based on the current solution configurations and saves them in the form of a special document. Grafana allows the exportation of diagrams for a defined time interval using REST API (If you’re interested in API performance testing consider PFLB as an advanced API load testing tool.)

Another module, called ResultToXLSX, defines the test phases based on the number of users. It also defines the load, compares it with the load profile, and calculates the 90th percentile based on the response time for each test transaction. The module exports the results as a table.

The results of the performance testing are compiled into an archive that is sent to the ATM testing engineers’ email. The engineers analyze the test and write a report.

The Project’s Future

We are planning to carry out the first automated analysis. Some work has already been done for this phase. We can already quickly check the performance automation metrics and compare the results with predefined values.

We also plan to automate the creation and modification of cases, as this is relatively time-consuming for some cases, such as when finding the maximum for some systems.

Finally, the performance of calculating VUsers in the load profile can be improved. At the moment, we use an Excel table, but in the future, this step will be added to the Pipeline.

Advantages for the Client

To ensure that your system remains available to users under a variety of circumstances and to find out what factors may reduce its performance, you need to conduct performance testing with Jenkins. Tests should be done consistently so that you can be sure that even a small change will not have a negative impact on the entire IT infrastructure.

Undoubtedly, one of the most important steps in the project is the automation of manual Jenkins performance testing. It speeds up the system preparation for the release of tests and reduces the overall time for the project.

Finally, an organization’s use of performance testing automation shows the maturity of its approach to quality control. Today, this factor is critical to gaining market share.