At a glance
- Client: one of the largest commercial banks, a domestic leader by equity, assets, profit and credit cards in circulation
- System: the bank's core banking platform - retail module, management accounting, front office, corporate budget planning and data warehouse, handling thousands of users and millions of documents
- Started: summer 2017, and the partnership continues
- The blocker: standard automation tools could not recognise the application's main menu - the entry point to every business operation
- What we built: SATBIS, a custom C++ library that proxies ActiveX requests to reach menu objects and the toolbar
- Outcome: regression testing time and cost reduced, coverage increased, and a test automation centre of excellence established inside the bank
- Tools: Winium and Selenium, C++, Java with JNI, JUnit, JDBC, Jenkins, Allure
Why regression testing was becoming unaffordable
The core banking system's strength is its bank-statement tooling - and that is also what makes it expensive to maintain. Financial regulators change statement requirements periodically, so every quarterly update carries not only new functionality but statutory changes that must ship fast.
Figure 1. Product architecture
Those changes touch base functionality, so nothing can be deployed without full regression testing. Meanwhile the regression model kept growing, and verifying changes of that depth demanded senior specialised analysts. The cost of every release rose with the one before it - the classic point at which manual regression stops scaling.
Three problems that blocked ordinary automation
1. The main menu was invisible to automation tools. It is the entry point of every business operation, so this alone made conventional UI automation impossible. 2. Table contents were unreadable beyond the visible area - and tables carry most of the system's functionality. Reading a full table took an unpredictable amount of time. 3. The regression model was only partially documented. How the system behaves during particular business operations lived in the heads of a few specialised analysts.
What we built
SATBIS - the System for Automated Testing of Bank Information Systems - is a C++ library written for this project that proxies ActiveX requests to obtain information about menu objects and provide access to the toolbar. It is what made the untouchable part of the interface testable.
Figure 2. Solution architecture
Figure 3. Framework architecture
Around it: Winium (a free Selenium-based desktop automation tool) for UI, JDBC for database work, Java as the main language with JNI integrating SATBIS, JUnit as the framework, Jenkins for continuous integration, and Allure - adapted to the bank's needs - for reports.
Tests were written at two levels. UI tests simulate a real user filling forms and clicking elements. API tests call stored procedures directly and check the responses. Some scenarios exist at both levels deliberately: auxiliary steps unrelated to the business process under test were moved from the slow UI layer to the fast API layer working straight against the database. That single decision is what made full regression runs practical rather than theoretical.
Undocumented knowledge was captured, not worked around. Senior specialists ran functional testing alongside the bank's analysts, audited the existing test model, and transferred what only those analysts knew into the test model itself.
Tests that survive product changes
Automated scenarios specify not only the parameters used but how each parameter value is obtained - SQL query, data file, template-based generation or a hard-coded value. The result is parameterisable tests that are not tied to a specific banking product or screen layout, and keep working when the bank's product line changes.
For example, the test that opens a term deposit picks the deposit type at random from the available list, then derives the minimum and maximum amount, term, account number format and permitted currencies from the database for that type. One test therefore covers many products, and coverage grows without new work for the client.
Results
- Regression testing time reduced, with the bank able to expand and automate the test model itself without missing schedules or lowering product quality.
- Regression coverage increased - a direct consequence of parameterised tests running across product types.
- A test automation centre of excellence was established at the bank, along with the technical framework for running the testing process in-house.
- PFLB supported deployment on the bank's own infrastructure and provided technical support for the delivered solution.
Questions this engagement answers
What do you do when automation tools cannot read the application?
Build the missing piece. Here that meant a C++ library proxying ActiveX calls to reach the main menu - a finite piece of engineering that unblocked everything downstream.
How do you automate against an undocumented regression model?
Pair with the analysts who hold the knowledge and move it into the test model as you go. The alternative - automating what is written down - automates the wrong half of the system.
How do you keep automated tests from breaking on every product change?
Parameterise them against the database rather than the screen: choose the product type at runtime and derive its rules from data, so one test covers a product line rather than a product.
Can the client take the process over?
That was the deliverable here - a centre of excellence inside the bank, not a permanent dependency on the vendor.
Regression costs rising with every release?
If each quarterly update costs more to verify than the last, the problem is not the size of your test suite - it is that the suite cannot run without your most senior people.
