Performance Testing of Blue Prism Intelligent Robotic Process Automation for Banking
Constant IT development is the new black. Many enterprises have already started using software designed to automate corporate processes with the help of AI. We share our favorite case of testing intelligent robotic process automation (RPA): Blue Prism, a platform providing a digital workforce that can perform any task of an operation staff member working on their PC. Whether they need to order stationery supplies or calculate salaries, Blue Prism virtual employees easily emulate business processes as real users would, regardless of the software used. Moreover, Blue Prism RPA is equipped with AI capabilities that make a lot of sense for the banking sector.
Table of Contents
1. RPA and AI for financial services
1.1. What is robotic process automation in banking?
1.2. How is RPA used in banking?
1.3. What is intelligent automation?
2. General performance testing tasks that the bank faced
3. Testing and scaling challenges and other RPA issues in the case
4. Process automation as a testing solution
4.1. Performance testing method
4.2. Connection between the app server and virtual users
4.3. Connecting an app server to the database
4.4. Features of the testing process
4.5. What we developed
5. Results of testing intelligent automation software for banking
RPA and AI for financial services
What is robotic process automation in banking?
Banking and financial services are the top sectors where RPA solutions are implemented. RPA helps cut costs, reduce response times, and—most importantly—free staff from routine tasks so that they can focus on more creative and innovative tasks. It has become even more popular due to the COVID-19 pandemic, as one of the problems faced by businesses during numerous lockdowns has been how to increase accessibility to on-site systems without crowding their offices.
How is RPA used in banking?
Financial organizations mainly use RPA for document automation in financial products, fund transfers, customer verification and requests, and data processing and verification. The list makes it obvious why RPA is so popular with banks.
What is intelligent automation?
Intelligent RPA is another step up toward automation. It unifies human and digital resources since it is equipped with artificial intelligence (AI)—and AI is a quick learner. Blue Prism RPA offers such solutions for the financial sector. However, although the company claims that its digital workforce operates 24/7, never makes mistakes, and has a 100% audit trail to account for the work they’ve done, our client faced several challenges and asked us to help.
The client is the biggest bank in Europe, with 98 million individual clients, 2.7 million corporate clients, and 278,000 employees. Although they had been using Blue Prism RPA for a while, they decided to perform the first ever performance testing of their software, so it was all new to them. PFLB played the guiding part, making the client aware of all the details of performance testing and running the service. After the project, the client’s business indicators increased notably, so they returned to us with another service request.
General performance testing tasks that the bank faced
01
Maximum number of digital workers that could be employed and processes that could be executed by Blue Prism RPA within the current system configuration.
02
Correlation between maximum productivity and system servers.
03
System bottlenecks.
04
Cases that could lead to system failures.
Testing and scaling challenges and other RPA issues in the case
The client gave us the results of a load testing of the system that they performed using their own capacities. Unfortunately, they were of no use at all.
First, the data was out of date, as processes tend to update on a regular basis. The test results were valid only at the moment of generation. The connection of new regional banks and processes to the system changes the load testing profile, which can critically affect system performance.
Second, test indicators directly depend on the complexity of the processes executed, and the testing results cannot be solid without a specific load. For instance, if a process creates and consumes queue elements saved in a database, it generates a much heavier load on application servers and the database, as opposed to supporting processes that work autonomously.
Another problem was that the client had prepaid 1,000 digital workers, while the system had a maximum capacity of 500. When around 400 Blue Prism virtual employees were active simultaneously, the system load reached 100%, and it would no longer respond.
Obviously, our client’s experience shows that it is recommended that you check your system capacity before buying software. Professional advice is more useful than the seller’s promise that your system will endure twice its maximum load.
We also had to decide how to scale, either by increasing the system servers’ capabilities or by scaling the architecture—that is, by deploying additional instances of the system.
Want to learn more about our performance testing services?
Find out what’s included and how
to start working with us
Process automation as a testing solution
Performance testing method
The client’s system comprised one database (DB) server, 14 app servers, one load balancer, and 700 Blue Prism robots. We used five servers for testing, including load generators. We also considered creating an emulator of virtual automated workers that would perform tasks instead of 700 digital workers. The load was supposed to be triggered upon receiving a signal sent by the virtual worker emulator. However, after analyzing the situation, we opted for a better solution. We decided to use JMeter, as it replaced digital workers with virtual users, or vusers, in a 1:1 ratio. After all, the only task of the virtual workers was to send requests to the app server. As it is very easy to increase the number of vusers or threads in JMeter, we managed to scale the load by changing a single setting.
Connection between the app server and virtual users
At the beginning, we had no information on communication between virtual users and the server, so it was a black box. We decided to record the traffic. Through our research, we found out that the system was using a .NET Remoting secure protocol. To simplify the process, we changed the performance testing mode to insecure and recorded traffic with Wireshark since .NET Remoting secure uses TCP requests. Basically, TCP requests are structured around the connection of the address, the quantity of bytes being sent, the body as a byte string, and the sign of request end. The traffic analyzer output was a hex string of the sent bytes.
At first, we used a self-written converter for parameterization. Later, we also read and controlled hex stings ourselves. In the end, the load script consisted of a package with TCP samplers, where the body was a hex string. Additionally, the script included preprocessors for parameterization and postprocessors for correlation.
Surprisingly, every TCP request with a valid structure received code 200 as a response, so the text content did not matter. To verify the response body, we used JMeter Assertions.
In the end, we created a tool capable of emulating the load generated by real digital workers and sending it to the Blue Prism RPA servers.
Connecting an app server to the database
We were provided with one app server for the testing process, although in reality there were 14 in the system. This was not a problem. Since the SQL request is transferred via protocol TDS, we recorded and replicated it. One JMeter vuser represented one app server by emulating the necessary traffic.
Features of the testing process
We reached a 100% load in the testing profile in the first run. However, the system resources seemed much less loaded than in real life. We identified several reasons for this:
- Unaccounted traffic
Processes are stored in the database as XML files that are sent to the digital worker by the app server at launch. When configuring the load profile, we took every file size into account. However, processes can be interconnected, in which case the vuser receives a package of XML-dependent processes at launch. This was substantially increasing the load on the network and the central processor of the database server.
- Storage of local variables
Process developers had stored local variables in a table with global variables in the database. Obviously, every vuser executing a process requested all the data from the table several times.
- Unsuccessful iterations
The executing process receives tasks using a procedure that is stored in the database. It turned out that of 233 procedure requests, new tasks were only received in 2% of the cases. Reproduction of this process resulted in a 20% increase in central processor utilization on the DB server.
- User interface (UI)
A large number of system administrators using the UI created additional loads on the servers.
- New protocol with disabled encoding
We estimated that 10% of the central processor’s load went to encoding messages.
What we developed
01
A script that emulates a Blue Prism digital workforce. This helped us determine the maximum performance of one Blue Prism server.
02
A script that emulates a Blue Prism server. This allowed us to recreate the necessary load on the database server while being short of Blue Prism servers.
03
Basic hardware monitoring for the performance testing profile (Telegraph, Influx, Grafana).
04
Self-written monitoring of operation response time based on constantly updated JMeter logs.
Results of testing intelligent automation software for banking
While conducting performance testing of intelligent automation software Blue Prism RPA for our client, a major financial service provider, we achieved the following results:
- We achieved a 100% system load that utilized almost the entire broadband capability (9 out of 10 Gbps). Still, a significant amount of other hardware resources remained intact.
- We determined optimal system configurations for different numbers of digital workers.
- We provided recommendations on how to optimize the scripts for digital workers.
- As the number of digital workers executing processes and processing tasks increased, hardware utilization grew exponentially. Expanding the broadband was not a solution since it wouldn’t have led to a significant increase in performance.
- We determined that the optimal solution to increase performance was horizontal system rescaling.
- Our recommendations were appreciated, and the client created two more test stages after the end of the project.
- The client was happy with the project results and requested that PFLB take on another testing project, the object being their own RPA system.
- During performance testing of Blue Prism RPA, our team applied and broadened its expert knowledge of working with systems based on the .NET Remoting protocol.
