Siebel CRM Load Testing

This happened during a transformation, in which they expected demand for the Bank’s products to increase, leading to more user operations and higher load on the system. As a result, we needed to use load testing to determine the Siebel system’s performance during peak season.

The system being tested is a credit approval system based on Siebel CRM. The system provides functionality for working with card and credit products, making it possible to conduct the full range of credit-servicing operations. The system also features a contact center module.

We accepted the challenge and decided to take a create approach to the problem. To begin, we studied how users work with the system. This turned out to be not so simple, since real user scenarios involve many stages. In fact, we had to build diagrams showing status transitions for requests and use statistics to determine the percentage of transitions at each stage.

To analyze user behavior, we had to crunch statistics and build different load profiles depending on the time of day and peak periods.

All the profiles and a description of the load model were summarized in a detailed document — the load testing methodology.

Determining the Load Testing Tool

Simultaneously with the methodology, we began to work out the approach to writing scripts and choose the most suitable tool for the system.

In the process of creating the load testing model for Oracle Siebel, we encountered a large number of difficulties that significantly increase the cost of testing.

When you’re deploying a live system, it is constantly receiving a huge number of improvements, fixes for blocking bugs, layout fixes, patches, and hot fixes. And everything would be fine, but with Siebel there is one nuance: the traffic is extremely sensitive to changing the interface, which means that, in addition to the fact that the scripts are extremely difficult to write due to the huge number of element correlations and identifiers, they almost instantly lose their relevance and become invalid.

For any action, a large amount of traffic is created in the system (more than 90 requests for opening an arbitrary page)

Existing systems (JMeter, LoadRunner) cannot determine which parameters need to be passed between requests. As a result, all traffic must be processed manually.

To solve this problem, PFLB created a FiddlerToJMeter plugin that makes it possible to automate the routine operations of recording and parameterizing the Siebel system’s traffic.

The tester only needs to specify which values to parameterize. In a single click, the plugin then prepares a ready-to-run JMeter script with recorded traffic.

Thus, the time spent on creating load scripts is significantly reduced, reducing the cost of testing without sacrificing quality.

Developing emulators for external systems

To emulate integration, we decided to use a test environment for functional testing, on which the external system was deployed.

But after a series of tests, the test bench proved to be weak for the external system and could not load Siebel. So we developed an external system emulator that handles requests with delays characteristic of the production environment.

Results of the Project

The customer was satisfied, and we gained good experience in testing systems based on Siebel CRM and a tool for fast correlation of complex traffic in JMeter scripts.

This plugin is now part of our load testing tool PFLB. So, if you have a problem that involves testing Siebel-based systems, we can help.