Performance Testing of Blue Prism Intelligent Robotic Process Automation for Banking

Constant IT development is the new black. Many enterprises have already started using software designed to automate corporate processes with the help of AI. We share our favorite case of testing intelligent robotic process automation (RPA): Blue Prism, a platform providing a digital workforce that can perform any task of an operation staff member working on their PC. Whether they need to order stationery supplies or calculate salaries, Blue Prism virtual employees easily emulate business processes as real users would, regardless of the software used. Moreover, Blue Prism RPA is equipped with AI capabilities that make a lot of sense for the banking sector.

RPA and AI for Financial Services

What is robotic process automation in banking?

Banking and financial services are the top sectors where RPA solutions are implemented. RPA helps cut costs, reduce response times, and—most importantly—free staff from routine tasks so that they can focus on more creative and innovative tasks. It has become even more popular due to the COVID-19 pandemic, as one of the problems faced by businesses during numerous lockdowns has been how to increase accessibility to on-site systems without crowding their offices.

How is RPA used in banking?

Financial organizations mainly use RPA for document automation in financial products, fund transfers, customer verification and requests, and data processing and verification. The list makes it obvious why RPA is so popular with banks.

What is intelligent automation?

Intelligent RPA is another step up toward automation. It unifies human and digital resources since it is equipped with artificial intelligence (AI)—and AI is a quick learner. Blue Prism RPA offers such solutions for the financial sector. However, although the company claims that its digital workforce operates 24/7, never makes mistakes, and has a 100% audit trail to account for the work they’ve done, our client faced several challenges and asked us to help.

The client is the biggest bank in Europe, with 98 million individual clients, 2.7 million corporate clients, and 278,000 employees. Although they had been using Blue Prism RPA for a while, they decided to perform the first ever performance testing of their software, so it was all new to them. PFLB played the guiding part, making the client aware of all the details of performance testing and running the service. After the project, the client’s business indicators increased notably, so they returned to us with another service request.

General Performance Testing Tasks That the Bank Faced

Our client wanted to increase the amount of digital workforce and obtain additional functions. PFLB was given the daunting and unusual task of performance testing a unique system, Blue Prism RPA, doing the research, and providing the client with the following data:

Testing and Scaling Challenges and Other RPA Issues in the Case

The client gave us the results of a load testing of the system that they performed using their own capacities. Unfortunately, they were of no use at all.

Obviously, our client’s experience shows that it is recommended that you check your system capacity before buying software. Professional advice is more useful than the seller’s promise that your system will endure twice its maximum load.

We also had to decide how to scale, either by increasing the system servers’ capabilities or by scaling the architecture—that is, by deploying additional instances of the system.

Process Automation as a Testing Solution

Performance testing method

The client’s system comprised one database (DB) server, 14 app servers, one load balancer, and 700 Blue Prism robots. We used five servers for testing, including load generators. We also considered creating an emulator of virtual automated workers that would perform tasks instead of 700 digital workers. The load was supposed to be triggered upon receiving a signal sent by the virtual worker emulator. However, after analyzing the situation, we opted for a better solution. We decided to use JMeter, as it replaced digital workers with virtual users, or vusers, in a 1:1 ratio. After all, the only task of the virtual workers was to send requests to the app server. As it is very easy to increase the number of vusers or threads in JMeter, we managed to scale the load by changing a single setting.

Connection between the app server and virtual users

At the beginning, we had no information on communication between virtual users and the server, so it was a black box. We decided to record the traffic. Through our research, we found out that the system was using a .NET Remoting secure protocol. To simplify the process, we changed the performance testing mode to insecure and recorded traffic with Wireshark since .NET Remoting secure uses TCP requests. Basically, TCP requests are structured around the connection of the address, the quantity of bytes being sent, the body as a byte string, and the sign of request end. The traffic analyzer output was a hex string of the sent bytes.

At first, we used a self-written converter for parameterization. Later, we also read and controlled hex stings ourselves. In the end, the load script consisted of a package with TCP samplers, where the body was a hex string. Additionally, the script included preprocessors for parameterization and postprocessors for correlation.

Surprisingly, every TCP request with a valid structure received code 200 as a response, so the text content did not matter. To verify the response body, we used JMeter Assertions.

In the end, we created a tool capable of emulating the load generated by real digital workers and sending it to the Blue Prism RPA servers.

Connecting an app server to the database

We were provided with one app server for the testing process, although in reality there were 14 in the system. This was not a problem. Since the SQL request is transferred via protocol TDS, we recorded and replicated it. One JMeter vuser represented one app server by emulating the necessary traffic.

Features of the testing process

We reached a 100% load in the testing profile in the first run. However, the system resources seemed much less loaded than in real life. We identified several reasons for this:

Processes are stored in the database as XML files that are sent to the digital worker by the app server at launch. When configuring the load profile, we took every file size into account. However, processes can be interconnected, in which case the vuser receives a package of XML-dependent processes at launch. This was substantially increasing the load on the network and the central processor of the database server.

Process developers had stored local variables in a table with global variables in the database. Obviously, every vuser executing a process requested all the data from the table several times.

The executing process receives tasks using a procedure that is stored in the database. It turned out that of 233 procedure requests, new tasks were only received in 2% of the cases. Reproduction of this process resulted in a 20% increase in central processor utilization on the DB server.

A large number of system administrators using the UI created additional loads on the servers.

We estimated that 10% of the central processor’s load went to encoding messages.

What we developed

Results of Testing Intelligent Automation Software for Banking

While conducting performance testing of intelligent automation software Blue Prism RPA for our client, a major financial service provider, we achieved the following results:

• We achieved a 100% system load that utilized almost the entire broadband capability (9 out of 10 Gbps). Still, a significant amount of other hardware resources remained intact.
• We determined optimal system configurations for different numbers of digital workers.
• We provided recommendations on how to optimize the scripts for digital workers.
• As the number of digital workers executing processes and processing tasks increased, hardware utilization grew exponentially. Expanding the broadband was not a solution since it wouldn’t have led to a significant increase in performance.
• We determined that the optimal solution to increase performance was horizontal system rescaling.
• Our recommendations were appreciated, and the client created two more test stages after the end of the project.
• The client was happy with the project results and requested that PFLB take on another testing project, the object being their own RPA system.
• During performance testing of Blue Prism RPA, our team applied and broadened its expert knowledge of working with systems based on the .NET Remoting protocol.