Go back to all articles

Contact Сenter and IVR Performance Testing

May 28, 2021
6 min read

Contact centers have replaced traditional call centers. Who wants to waste human resources on something that can be easily managed by software? Today, contact centers are an integral part of retail and service providers.

Unlike call centers, which can only interact with customers via the public switched telephone network (PSTN), contact centers include services that improve customer support and offer a wide range of tools for the most convenient communication and interaction.

The capabilities of basic telephone interaction have expanded thanks to interactive voice response (IVR). Managing customer requests in automatic mode and using speech recognition algorithms and smart navigation are only two of the methods that have made it possible to automate the process of customer interaction to a great extent.

Obviously, the higher the complexity of a contact center’s structure, the higher the level of hardware, and the higher the likelihood of unsatisfactory performance of the system or its components.

Given that the quality of the services provided through a contact center directly affects user experience and, as a result, consumer attitude toward a product, service, or company as a whole, there is hardly anything more important than the timely identification of possible problems.

Contact Centers Performance Issues

The central issue behind contact center productivity is the technical capabilities of a company’s IT infrastructure. If they are insufficient to meet your business’s needs, you can end up with additional costs and unhappy customers.

The top 4 IVR performance problems are:

  • poor sound quality;
  • audio signal delay;
  • poor or interrupted connection;
  • slow IVR navigation response.

If one or more of these problems are present, customers may misunderstand the message and hold false expectations or may have to ask the assistant to repeat the message, which would increase the total duration of the call. Either scenario would result in a poor customer experience.

Why is call duration so crucial?

Obviously, with an increase in the average duration of calls, the number of operators required to process them also increases. For example, a typical call lasting 3 minutes and 15 seconds can increase by 12 seconds if there are communication issues. The cumulative effect is equivalent to an additional 6 contact center operators for every 100.

ivr load testing 1

Load and communication quality

Perceptual Evaluation of Speech Quality (PESQ), an international standard for objective voice quality testing, measures voice transmission quality using a mean opinion score (MOS):

MOSQualityDistortion

5

Excellent

Imperceptible

4

Good

Noticeable but not annoying

3

Ok

Somewhat annoying

2

Bad

Annoying

1

Terrible

Very annoying

The lower the MOS, the worse the voice quality is perceived and the more difficult it is for customers to interact with operators.

Although some performance problems are obvious, mere observations without measurements of the overall load level and system functioning quality are not enough, especially when it comes to a large customer base or wide coverage.

On a regular business day, the call flow gradually increases. Accordingly, the quality of voice transmission gradually deteriorates. In the model graph below, you can see what can happen at 6 pm. A sale in retail, or any emergency, leads to an increased flow of calls and, ultimately, to system failure.

a call flow increase leads to a deterioration in the quality of voice transmission 1

Situations like this can arise from a wide range of IVR performance problems that cannot be identified in advance without load testing.

Testing Contact Center IVR with LoadRunner and Data Synergy Voice

Load testing is an effective solution to the abovementioned problems. If you don’t want to waste time, it is important to run the tests.

PFLB specialists use some of the best online load testing tools: Open Text’s well-known LoadRunner and the DS Voice plugin developed by our partner, Data Synergy. LoadRunner emulates thousands of user interactions with the tested system, while DS Voice interacts with digital telephony communication channels.

ivr load testing data 1

You can emulate and process interactions between customers and contact center operators using LoadRunner and DS Voice. In particular, you can:

  • generate incoming and outgoing voice traffic over SIP channels;
  • measure the quality of voice communication (delays, distortion, packet loss);
  • transmit and process DTMF codes;
  • record, play, and recognize an audio stream;
  • convert text to audio.
ivr load testing scheme 1

With our IVR system testing solution, you will be able to:

  • identify problems that users face during busy hours;
  • analyze a wide range of application and infrastructure parameters under different load levels;
  • find performance bottlenecks at all architectural levels;
  • measure the scalability parameters of the contact center;
  • reate a load both from inside and outside the contact center’s local network.

IVR Performance Testing: Our Approach

Modern contact centers are often complex systems that involve close interaction between IVR, customer relation management (CRM), and many other services. Therefore, besides testing the performance of IVR itself and telephone communication channels, PFLB offers comprehensive testing services that allow you to evaluate the performance of both overall and individual services and to identify the influence of system components on each other.

ivr load testing scheme data synergy 1

How do we test IVR systems?

There are four necessary stages plus an optional one:

  • 01
    developing an IVR testing methodology;
  • 02
    developing IVR performance testing tools;
  • 03
    running the tests;
  • 04
    analyzing the system and compiling a report;
  • 05
    retesting (optional).
Have a Project in Mind?​
We have been working on performance testing projects since 2008.
Drop us a line to find out what our team can do for you.
Get a quote You’ll hear back from our tech account manager in one day if not sooner

1. Developing an IVR testing methodology

PFLB specialists create the testing methodology after analyzing the technical documentation, business processes configured in the IVR, and contact center platforms, as well as interviewing the client company’s specialists. What we do is as follows:

  • collect and analyze industrial environment statistics;
  • determine load generation methods and generator locations;
  • coordinate performance requirements;
  • define business processes and load scenarios;
  • describe interactions with external systems.

Then, we present the chosen methods to the client for approval.
When creating the testing method, we pay special attention to determining the load profile. The load profile is a set of operations with specified intensities that can be determined based on statistical data or an analysis of the system requirements. As a rule, several load profiles are selected, each of which differs in the set of operations or intensity. The greater the load on the system, the higher the quality of the load testing.

2. Developing IVR performance testing tools

Creating a test model includes six steps:

  • 01
    developing load scripts and load scenarios;
  • 02
  • 03
    creating scripts;
  • 04
    generating/depersonalizing the database (if necessary);
  • 05
    preparing test data;
  • 06
    configuring monitoring tools and running trial tests.

This is the most challenging stage from a technical point of view. PFLB engineers develop and debug the necessary IVR performance testing tools.

As part of this stage, PFLB offers its clients the opportunity to use its in-house technology, PLUS, which can help cut costs and improve the testing quality using ready-made performance testing tools, such as Data Synergy Voice, SOAP/REST processing emulator ISO 8583, SOAP emulator of a credit broker, WebSphere log analyzer, MS SQL database depersonalization module, and Visa and Mastercard clearing test data generator.

3. Running the tests

This stage comprises running the tests according to the approved load scenarios, processing the results, and evaluating the tests’ validity. PFLB specialists are given access to load generation stations during the testing.

4. Analyzing the system and compiling a report

PFLB specialists analyze the results and share their conclusions and recommendations with the client. They pay special attention to finding performance bottlenecks, preparing recommendations on optimizing the code, architecture, and infrastructure of the system, or developing appropriate organizational regulations. This is when they also determine whether the channel width is sufficient for the IVR, the operator’s real response time, the level of communication quality, etc. Then, they compile a report with the test results, detected problems, and recommendations.

5. Retesting (optional)

After the client performs the recommended optimizations, tests can be run again to confirm the system performance improvement and identify any new factors that limit performance.

We can also develop a plan for the client to keep the performance testing solutions up-to-date for future retests.

PFLB offers services for setting up the support process, which can include both comprehensive maintenance of the created solution and training of the client’s staff engineers.

About the Company

PFLB is a Delaware-based, leading performance testing company working globally with enterprise clients. The company has successfully carried out over 300 major projects and is SOC 2 Type 2 certified.

PFLB employs more than 400 specialists, and its clients include many of the biggest banks, retail companies, telecommunications providers, and many other organizations. Clients that have used PFLB’s IVR load testing with DS Voice include IKEA and The Coca-Cola Company.

Table of contents

Related insights in blog articles

Explore what we’ve learned from these experiences
7 min read

SRE Roles and Responsibilities: Key Insights Every Engineer Should Know

sre roles and responsibilities preview
Sep 11, 2024

Site Reliability Engineers (SREs) are crucial for maintaining the reliability and efficiency of software systems. They work at the intersection of development and operations to solve performance issues and ensure system scalability. This article will detail the SRE roles and responsibilities, offering vital insights into their duties and required skills. Key Takeaways Understanding Site Reliability […]

11 min read

Understanding Error Budgets: What Is Error Budget and How to Use It

understanding error budgets what is error budget and how to use it preview
Sep 10, 2024

An error budget defines the allowable downtime or errors for a system within a specific period, balancing innovation and reliability. In this article, you’ll learn what is error budget, how it’s calculated, and why it’s essential for maintaining system performance and user satisfaction. Key Takeaways Understanding Error Budgets: What Is Error Budget and How to […]

10 min read

Mastering Reliability: The 4 Golden Signals SRE Metrics

mastering reliability the 4 golden signals sre metrics preview
Sep 9, 2024

Introduction to Site Reliability Engineering Site Reliability Engineering is a modern IT approach designed to ensure that software systems are both highly reliable and scalable. By leveraging data and automation, SRE helps manage the complexity of distributed systems and accelerates software delivery. A key aspect of SRE is monitoring, which provides real-time insights into both […]

9 min read

Reliability vs Availability: Key Differences

reliability vs availability key differences preview
Sep 6, 2024

Defining Reliability and Availability What is Reliability? Reliability refers to the probability that a system will consistently perform as expected, delivering correct output over a set period of time. In the world of Site Reliability Engineering (SRE), reliability is a core metric that drives everything we do. It’s not just about whether a service works […]

  • Be the first one to know

    We’ll send you a monthly e-mail with all the useful insights that we will have found and analyzed