Go back to all articles

Performance Testing of Blue Prism Intelligent Robotic Process Automation for Banking

Apr 29, 2021
10 min read

Constant IT development is the new black. Many enterprises have already started using software designed to automate corporate processes with the help of AI. We share our favorite case of testing intelligent robotic process automation (RPA): Blue Prism, a platform providing a digital workforce that can perform any task of an operation staff member working on their PC. Whether they need to order stationery supplies or calculate salaries, Blue Prism virtual employees easily emulate business processes as real users would, regardless of the software used. Moreover, Blue Prism RPA is equipped with AI capabilities that make a lot of sense for the banking sector.

RPA and AI for Financial Services

What is robotic process automation in banking?

Banking and financial services are the top sectors where RPA solutions are implemented. RPA helps cut costs, reduce response times, and—most importantly—free staff from routine tasks so that they can focus on more creative and innovative tasks. It has become even more popular due to the COVID-19 pandemic, as one of the problems faced by businesses during numerous lockdowns has been how to increase accessibility to on-site systems without crowding their offices.

How is RPA used in banking?

Financial organizations mainly use RPA for document automation in financial products, fund transfers, customer verification and requests, and data processing and verification. The list makes it obvious why RPA is so popular with banks.

What is intelligent automation?

Intelligent RPA is another step up toward automation. It unifies human and digital resources since it is equipped with artificial intelligence (AI)—and AI is a quick learner. Blue Prism RPA offers such solutions for the financial sector. However, although the company claims that its digital workforce operates 24/7, never makes mistakes, and has a 100% audit trail to account for the work they’ve done, our client faced several challenges and asked us to help.

blue prism scheme

The client is the biggest bank in Europe, with 98 million individual clients, 2.7 million corporate clients, and 278,000 employees. Although they had been using Blue Prism RPA for a while, they decided to perform the first ever performance testing of their software, so it was all new to them. PFLB played the guiding part, making the client aware of all the details of performance testing and running the service. After the project, the client’s business indicators increased notably, so they returned to us with another service request.

General Performance Testing Tasks That the Bank Faced

Our client wanted to increase the amount of digital workforce and obtain additional functions. PFLB was given the daunting and unusual task of performance testing a unique system, Blue Prism RPA, doing the research, and providing the client with the following data:

  • 01
    Maximum number of digital workers that could be employed and processes that could be executed by Blue Prism RPA within the current system configuration.
  • 02
    Correlation between maximum productivity and system servers.
  • 03
    System bottlenecks.
  • 04
    Cases that could lead to system failures.

Testing and Scaling Challenges and Other RPA Issues in the Case

The client gave us the results of a load testing of the system that they performed using their own capacities. Unfortunately, they were of no use at all.

  • First, the data was out of date, as processes tend to update on a regular basis. The test results were valid only at the moment of generation. The connection of new regional banks and processes to the system changes the load testing profile, which can critically affect system performance.
  • Second, test indicators directly depend on the complexity of the processes executed, and the testing results cannot be solid without a specific load. For instance, if a process creates and consumes queue elements saved in a database, it generates a much heavier load on application servers and the database, as opposed to supporting processes that work autonomously.
  • Another problem was that the client had prepaid 1,000 digital workers, while the system had a maximum capacity of 500. When around 400 Blue Prism virtual employees were active simultaneously, the system load reached 100%, and it would no longer respond.

Obviously, our client’s experience shows that it is recommended that you check your system capacity before buying software. Professional advice is more useful than the seller’s promise that your system will endure twice its maximum load.

We also had to decide how to scale, either by increasing the system servers’ capabilities or by scaling the architecture—that is, by deploying additional instances of the system.

Want to Learn More About Our Performance Testing Services?
Find out what’s included and how to start working with us.

Process Automation as a Testing Solution

Performance testing method

The client’s system comprised one database (DB) server, 14 app servers, one load balancer, and 700 Blue Prism robots. We used five servers for testing, including load generators. We also considered creating an emulator of virtual automated workers that would perform tasks instead of 700 digital workers. The load was supposed to be triggered upon receiving a signal sent by the virtual worker emulator. However, after analyzing the situation, we opted for a better solution. We decided to use JMeter, as it replaced digital workers with virtual users, or vusers, in a 1:1 ratio. After all, the only task of the virtual workers was to send requests to the app server. As it is very easy to increase the number of vusers or threads in JMeter, we managed to scale the load by changing a single setting.

Connection between the app server and virtual users

At the beginning, we had no information on communication between virtual users and the server, so it was a black box. We decided to record the traffic. Through our research, we found out that the system was using a .NET Remoting secure protocol. To simplify the process, we changed the performance testing mode to insecure and recorded traffic with Wireshark since .NET Remoting secure uses TCP requests. Basically, TCP requests are structured around the connection of the address, the quantity of bytes being sent, the body as a byte string, and the sign of request end. The traffic analyzer output was a hex string of the sent bytes.

At first, we used a self-written converter for parameterization. Later, we also read and controlled hex stings ourselves. In the end, the load script consisted of a package with TCP samplers, where the body was a hex string. Additionally, the script included preprocessors for parameterization and postprocessors for correlation.

Surprisingly, every TCP request with a valid structure received code 200 as a response, so the text content did not matter. To verify the response body, we used JMeter Assertions.

In the end, we created a tool capable of emulating the load generated by real digital workers and sending it to the Blue Prism RPA servers.

Connecting an app server to the database

We were provided with one app server for the testing process, although in reality there were 14 in the system. This was not a problem. Since the SQL request is transferred via protocol TDS, we recorded and replicated it. One JMeter vuser represented one app server by emulating the necessary traffic.

Features of the testing process

We reached a 100% load in the testing profile in the first run. However, the system resources seemed much less loaded than in real life. We identified several reasons for this:

  • Unaccounted traffic

Processes are stored in the database as XML files that are sent to the digital worker by the app server at launch. When configuring the load profile, we took every file size into account. However, processes can be interconnected, in which case the vuser receives a package of XML-dependent processes at launch. This was substantially increasing the load on the network and the central processor of the database server.

  • Storage of local variables

Process developers had stored local variables in a table with global variables in the database. Obviously, every vuser executing a process requested all the data from the table several times.

  • Unsuccessful iterations

The executing process receives tasks using a procedure that is stored in the database. It turned out that of 233 procedure requests, new tasks were only received in 2% of the cases. Reproduction of this process resulted in a 20% increase in central processor utilization on the DB server.

  • User interface (UI)

A large number of system administrators using the UI created additional loads on the servers.

  • New protocol with disabled encoding

We estimated that 10% of the central processor’s load went to encoding messages.

What we developed

  • 01
    A script that emulates a Blue Prism digital workforce. This helped us determine the maximum performance of one Blue Prism server.
  • 02
    A script that emulates a Blue Prism server. This allowed us to recreate the necessary load on the database server while being short of Blue Prism servers.
  • 03
    Basic hardware monitoring for the performance testing profile (Telegraph, Influx, Grafana).
  • 04
    Self-written monitoring of operation response time based on constantly updated JMeter logs.

Results of Testing Intelligent Automation Software for Banking

While conducting performance testing of intelligent automation software Blue Prism RPA for our client, a major financial service provider, we achieved the following results:

  • We achieved a 100% system load that utilized almost the entire broadband capability (9 out of 10 Gbps). Still, a significant amount of other hardware resources remained intact.
  • We determined optimal system configurations for different numbers of digital workers.
  • We provided recommendations on how to optimize the scripts for digital workers.
  • As the number of digital workers executing processes and processing tasks increased, hardware utilization grew exponentially. Expanding the broadband was not a solution since it wouldn’t have led to a significant increase in performance.
  • We determined that the optimal solution to increase performance was horizontal system rescaling.
  • Our recommendations were appreciated, and the client created two more test stages after the end of the project.
  • The client was happy with the project results and requested that PFLB take on another testing project, the object being their own RPA system.
  • During performance testing of Blue Prism RPA, our team applied and broadened its expert knowledge of working with systems based on the .NET Remoting protocol.
Table of contents
Let us know about your needs
We can provide multiple performance testing services and a lot more than that if the situation needs a far more complex approach.
Get a quote You’ll hear back from our tech account manager in one day if not sooner

Related insights in case studies

Explore what we’ve learned from these experiences
8 min read

How Load Testing Helped Texans Survive Power Outages During a Storm

how pflb helped texans survive power outages during a storm preview
Jun 13, 2024

One of the largest electric distribution company in Texas that serves over 100,000 people, ensuring an uninterrupted power supply. To maintain service quality during weather-related power outages, it must comply with the Texas Reliability Entity, which ensures energy providers’ reliability and security. To provide clients with reliable service, the company deployed Storm Center and the […]

4 min read

Tynor Prepared the New Website for High Sales in Four Days

tynor prepared the new website for high sales in four days preview
Dec 12, 2022

Tynor Orthotics is India’s largest manufacturer and exporter of orthopedic and fracture aids established in the 90s to deliver quality healthcare products. Committed to a significant expansion in the next three years, Tynor crafted a new e-commerce website focused on excellent customer experience to support this growth. To be confident at launch, the engineering team of Tynor decided to run pre-go-live stress testing for the website. The tight deadline felt challenging, the customer was relieved to hear we provide a quick load testing solution. Quick Load Testing solution includes a four-day load testing project performed by engineers of PFLB and a 1-month subscription to the innovative load testing PFLB platform.

5 min read

Bank Increases Load Capacity by 450% to Deal with Business Growth

bank increases load capacity by 450 to deal with business growth preview
Oct 3, 2022

Our client's bank was absorbing other banks, and the number of individual clients was growing. The system was not ready for expansion or integration. The owners started to suspect bottlenecks when problems with paying salaries to corporate clients’ employees arose. As a result, in the next pay period, the load on the system increased dramatically, and the system got overloaded. People did not get their salaries in time, as the system crashed.

5 min read

Testing Major Meat Producer’s SAP Makes System 8 Times Faster

testing major meat producers sap makes system 8 times faster
Jun 2, 2022

Our client, a major nation-wide meat producer, was actively using local WMS and TMS. As a result, their 5,000 employees used to waste much time and effort storing and processing a lot of data in different systems. To increase the effectiveness of the document flow, the client decided to move all the processes to one SAP ERP system. Massive migration was about to happen, since enormous amounts of data from different legacy systems had to be centralized. The client had extremely high requirements for a performance tuning project contractor. They wanted to entrust the task only to a highly experienced team. The choice fell on PFLB, as the company has a proven track record of delivering SAP load testing projects for many corporate clients around the world.

  • Be the first one to know

    We’ll send you a monthly e-mail with all the useful insights that we will have found and analyzed

  • People love to read

    Explore the most popular articles we’ve written so far