Bank Increases Load Capacity by 450% to Deal with Business Growth

How the Need for System Performance Testing Arose

The bank’s management made a strategic decision to develop their remote banking services in-house and to try and bring the system to a higher level in the ratings. Three platforms (online web banking, iOS and Android apps) were subject to renovation.

The situation was aggravated by the fact that at that moment the banking group was undergoing integration processes, and the load on the system increased in step-ups. It became clear that the system was not ready for such a load: there was a system crash on a peak workload day, a payday.

Since the remote banking services system operates in an environment of integration with related systems (accounting, active operations, sales channels, terminal networks, etc.), to identify the problem area, specialists need to evaluate the whole complex of these components.

In that case it was necessary to find out which system malfunctioned, what particular problems disrupted the work of the client Internet banking.

In order to deal more quickly with the critical task of finding the problem, the CIO and CTO of the bank decided to outsource the task.

Urgent Solution

The first step of the PFLB team was to issue several recommendations that helped the bank survive the next banking day without system failures.

Our best engineers immediately engaged in an investigation of the system overload. We found 200+ system errors that impaired both the system’s performance and its stability.

After having increased database session timeout from 5 seconds to 30, we made sure there will be no system crashes in the next payroll period by conducting performance tests right before it. Thus, we located one of the most critical bottlenecks and saved the immediate future of the bank’s clients. We also ran stress testing to be able to predict the behavior of the system from the next peak day and to see if it will stand the load.

At the same time, we actively collaborated with the bank’s staff to form a pool of recommendations needed to be implemented to stabilize the system. We helped to prioritize these tasks and began the process of removing bottlenecks. Overall, it took us less than 2 weeks.

The bank’s remote banking services system was under active development, which could have caused changes in system performance during the release of new versions. The client made a strategic decision to build and implement a performance testing process as a routine part of their business process to minimize the risks. We were eager to help, and that is how the second stage of the project started.

Sustainable Solution

Speaking strategically, our client wanted the system to have a load reserve that would ensure system stability with further user number increases. We worked on this stage of the project for half a year, because the bank system was expanding and required a full-fledged performance testing process to be designed, introduced and implemented.

During the second phase of the project, which lasted for six months, the following results were achieved:

Conclusions

The system now has load capacity of up to 450% of the production environment. Now the bank can freely perform critical business processes without a threat of facing a system crash.

During the project, the bank’s in-house team accumulated enough skills to be able to identify the causes and locations of bottlenecks, and thus, performance testing and localizing performance issues is since then the IT department’s function.

Since the bank’s policy was initially aimed at in-house dev, embedding performance testing into the development process was a valuable experience that increased the maturity of the bank’s development culture.