Go back to all articles

Synthetic Load Testing for PaaS Educational Project

Feb 5, 2020
8 min read

Business Challenge:

In the fall, a company assigned us a task of conducting load testing. It turned out that the customer was implementing an educational project based on the platform as a service (PaaS) principle and was creating a platform that would ensure effective interactions between teachers and students in the educational process.

The customer was faced with an ambitious task of launching a pilot project in general education schools in six regions of the country during six months and then replicating the project to general education schools.
Want to Learn More About Our Performance Testing Services?
Find out what’s included and how to start working with us.

The Customer Had the Following Choices:

  • 01
    Purchase a box solution and customize with subsequent refinement and integration.
  • 02
    Develop an in-house information system.

The customer chose the first option. They then purchased a solution for the pilot program from an American educational platform developer and enhanced the basic product to meet the established requirements.

The Main Task:

In load testing, there are different types of tests. For example:

  • Performance testing, i.e. identifying the maximum performance and determining the point of system failure.
  • Stress testing, i.e. analysis of the system under conditions that exceed the norm.
  • Stability testing, i.e. analysis of the system under a prolonged excessive load.
  • Volume testing, i.e. investigation of system performance with increasing amounts of data added to the application database.

The main task of load testing, in this case, was to confirm that the system met the stated requirements for maximum performance, which was defined as supporting as many as 500,000 simultaneous users.
To begin with, we suggested running a performance test consisting of one iteration of load testing to determine the maximum performance.

In order to run the test to find maximum performance, a number of preparatory tasks needed to be performed:​​

  • 01
    Describe the system architecture.
  • 02
    Define the characteristics of the testing stand and assemble it.
  • 03
    Define the list of business processes that will be included in the load testing profile.
  • 04
    Develop a load testing profile based on currently available data.
  • 05
    Write a load testing methodology and specify in detail the upcoming mechanism for load testing.

Architecture Description and Testing Stand Preparation

Preparation of the load testing stand is an integral and very important stage of testing. The ideal stand configuration is an identical copy of an industrial stand, but this is not always possible when testing large-scale systems. In this case, it was critical to choose an optimal configuration to minimize the utilized capacity and obtain data that could be reliably extrapolated to the industrial system.

In this case
PFLB engineers were very lucky, because the system was not in the industrial operation, and we had a rare opportunity to conduct testing in the prepared “battlefield environment”, and this was a big plus.

Testing Profile

However, the fact that the system was not in industrial operation was also a significant minus. It was impossible to collect user statistics about the system business processes. As a rule, in such situations, when a load profile is expertly compiled, the business customer and industry experts are involved. That said, we had an opportunity to access information from open sources and we used official data from Ministry of Education and Science.

Thus, after collecting and analyzing all of the available data, we had a load profile ready, the  requirements for filling the database were defined and the load testing stand was prepared.  We were ready to conduct load testing in the conditions which were as close to “battlefield operation” as possible.

Testing

From the very start of the project, we have been writing the methodology and preparing load testing facilities (LTF) in parallel. LTF are capacities that will generate load and to test them sufficiently, it was necessary to estimate the throughput of an individual load station. This stage was needed to calculate the number of load stations required to supply the target load.

The throughput of a load station is measured using synthetic testing, i.e. standardized tests showing the performance of an information technology (IT) system in terms of hardware and software performance metrics.

In the process of searching the throughput of load stations, the load testing team did not have access to testing stand machines, and we did not have an opportunity to observe the utilization of hardware resources of the stand itself. After the first synthetic test, it became clear that central processing unit (CPU) utilization at the load station did not exceed 20%. The first concern was the non-optimal performance of the load scripts. After a series of optimizations to the load scripts, we still observed no improvements.

The next possible reason for the throughput limitation was the network channel itself (in load scripts there was a large number of web statics). The load testing team transferred load stations to one network with a gigabit communication channel instead of 100 Mbps. After transferring to one network, a few more synthetic tests were performed which showed a similar result. The CPU load of the load station did not exceed 25% or 3% of the target load.

The reasons for the loading station side were excluded, and the team received access to testing stand machines in due time. Monitoring software to track hardware metrics was installed on the testing stand and the PFLB team continued synthetic testing.

During the test, it was revealed that, at 10% of the target load, the web server’s CPU utilization reached 100% and after the test was terminated that CPU utilization dropped to 2%. The customer was notified of this problem, and then the team needed to correct the test plan and load testing methodology as soon as possible.

Before the problems with the web server’s CPU utilization were identified, stages of 10% of the target intensity (from 10% to 110%) had been set in the methodology to find the maximum performance. In the current configuration, it was clear that starting with 10% intensity did not make sense. To solve this problem, we decided to introduce into the test plan “micro stages” of 0.5% of the load from the target load up to 10% (20 stages).

Results

  • The results of load testing showed that the given configuration for the system is far from the stated performance requirements. The maximum system performance was 2% of the target performance.
  • The reasons for the system performance limitations were localized and recommendations for their elimination were described in detail.
  • Within a short time and at an early stage the customer obtained insight into the system performance limits, localized bottlenecks and gained an understanding of the amount of work to be done in order to increase performance.
  • Since the testing was conducted at an early stage, the customer obtained the key data necessary to make a decision about the further development of the project. At this stage, replacement of the vendor/developer is acceptable, and the PFLB team is ready to provide support in eliminating constraints and repeat the test iteration.
Table of contents
Let us know about your needs
We can provide multiple performance testing services and a lot more than that if the situation needs a far more complex approach.
Get a quote You’ll hear back from our tech account manager in one day if not sooner

Related insights in case studies

Explore what we’ve learned from these experiences
7 min read

From Hundreds to Thousands: Scaling MEFA Pathway Software for Mass Student Registration

how load testing helped e learning services provider preview
Jul 12, 2024

FolderWave, Inc. is a leading digital services provider in the Massachusetts e-learning sector. It aids millions of students in researching and planning a job-oriented education. The company delivers IT solutions, platforms, and services in partnership with notable non-profit organizations like MEFA Pathway and College Board, which connect a vast network of colleges, schools, and universities […]

8 min read

How Load Testing Helped Texans Survive Power Outages During a Storm

how pflb helped texans survive power outages during a storm preview
Jun 13, 2024

Background The largest electric distribution cooperative in Texas and the United States, Pedernales Electric Cooperative (PEC), had to test its new software systems, the Storm Center and the OR&S (Outage Reporting & Status), before the release to ensure their adequate performance under peak load. Challenge PEC had a strict release deadline and needed to test […]

4 min read

Tynor Prepared the New Website for High Sales in Four Days

tynor prepared the new website for high sales in four days preview
Dec 12, 2022

Tynor Orthotics is India’s largest manufacturer and exporter of orthopedic and fracture aids established in the 90s to deliver quality healthcare products. Committed to a significant expansion in the next three years, Tynor crafted a new e-commerce website focused on excellent customer experience to support this growth. To be confident at launch, the engineering team of Tynor decided to run pre-go-live stress testing for the website. The tight deadline felt challenging, the customer was relieved to hear we provide a quick load testing solution. Quick Load Testing solution includes a four-day load testing project performed by engineers of PFLB and a 1-month subscription to the innovative load testing PFLB platform.

5 min read

Bank Increases Load Capacity by 450% to Deal with Business Growth

bank increases load capacity by 450 to deal with business growth preview
Oct 3, 2022

Our client's bank was absorbing other banks, and the number of individual clients was growing. The system was not ready for expansion or integration. The owners started to suspect bottlenecks when problems with paying salaries to corporate clients’ employees arose. As a result, in the next pay period, the load on the system increased dramatically, and the system got overloaded. People did not get their salaries in time, as the system crashed.

  • Be the first one to know

    We’ll send you a monthly e-mail with all the useful insights that we will have found and analyzed

  • People love to read

    Explore the most popular articles we’ve written so far