Project Management in Performance and Load Testing Projects

There are two models of providing performance testing services to the customer: outstaff and fixed price. If your customers choose outstaffing, they have an experienced testing engineer on staff. This person will be able to set goals and tasks for the testing process and act as a performance testing manager. In this case, they only pay your engineers’ hours and take full responsibility for the results.

A fixed price means that the responsibility is on your side. Your customer can’t run the tests themself and wants a package deal. As a performance testing manager, you must design everything from scratch. In this article, we’re looking at the stages of a fixed-price project, as seen by an IT project manager.

1. Find out As Much As Possible Before You Start the Project

If your company uses Confluence, by the time you are assigned a new project, some data on the project will probably already be there in the presale section. Before you start, take a look at the latest review. It might contain details on your QA team, tasks, and project stages. Download it if you have access, or ask your manager to give you one.

Another good idea is to study the contract. It would have timescales, benchmarks, and a basic list of tasks. Do this before you even start the project, too.

2. Design a Communication Map

The first thing you need to do with the customer is fill in your communication map. It should include all the stakeholders, such as the management, architects, developers, analysts, admins, consultants, and business. You will need architects because they know their system better than you will ever. You need developers for consultations and analysts to ask them about business processes.

Administrators may be the key to your testing environment. SAP consultants are very aware of how every business process runs. Access to each and every one of these specialists will make your work much easier.

3. Hold the First QA Team Meeting

Whether you have already drafted your map of communications or not, the first meeting is important for your QA project roadmap. If possible, ask someone from the customer’s side to organize it. While at it, use your survey or a sample of performance testing methodology to ask all the questions you need. If you feel like you have too many, start with the questions directly related to the first stage of the performance testing project, in case there will be no time for all of them. You must give tasks to your engineers after the meeting, so you must understand enough.

A little life hack is to record the meeting since you might miss something and want to go back to the recording many times later. You have to ask all participants for permission to start the recording.

4. Make a List of Business and Technical Goals for Your Performance Testing Project

Make sure the next step you take is identifying the list of business and technical project goals.
The business goals of a performance testing project equal its background causes. Basically, it is why the customer got the idea they needed performance testing. It may be business scaling, a new release, or getting ready for the Black Friday rush.

You will have to develop technical goals for performance testing as a performance testing manager based on your customers’ needs. The list may include eliminating bugs in the production environment, finding bottlenecks, decreasing system response time, defining maximal productivity, reliability testing, etc. Most often, the customer won’t know their technical goals, and you will have to translate their business needs into the technical goals of your project. For example, suppose your customer wants performance testing because of a system crash in production. To translate it into technical goals, you may recommend analyzing logs from that particular crash moment, making a profile based on those logs, looking for operations being processed then, and trying to reconstruct the situation in the testing environment.

You should, of course, base your recommendations on the overall project evaluation. Meeting time limits is more important than precisely following the scope of tasks since the latter can change a lot during the project. It doesn’t matter if the contract says you have to test maximum performance while the customer asks for integration testing or other tests. However, since project evaluation is based on the number of professionals and the time they will work on it, it is really important to meet the deadlines.

Remember that customers sometimes try to get a free ride and ask for much more than they have paid for. Your contract with the list of technical goals will save you from such problems.

5. Make the Customer Hand over Everything You Need to Start

The earlier you give the customer the list of what you need for the project, the earlier you receive it. It’s best to try to discuss it in the first meeting.

You will probably need direct contact with architects or an architectural layout they made. They know their product much better, so the latter is preferable.

You need to familiarize yourself with the system, its capacities, and the hardware it is based on. One of the main questions is how the testing environment corresponds to production. If, for instance, you are given one environment and then told to run tests in different testing environments, you’ll waste your time. And, as mentioned above, time is money.

You’ll need their statistics since you can’t start writing scripts without a load profile. You’ll need access to environments, hardware, and, most importantly, to your customer’s staff. After you’ve designed your communications map, you come up with questions or requests and deal with particular people on staff instead of the management. The person you need may tell you they are busy with their tasks. To avoid conflicts and/or holdback, it’s better if the management from the customer’s side can warn their staff about their involvement with your project and the reporting period.

6. Approach the Testing: Figure out What You Will Test, Sources of Load and Integration Details

Based on the architecture, you must decide which servers are subject to testing, what is in the system, how you will serve loads, etc. There are a great deal of factors to consider. The system is integrated, it sends and receives data, users interact with it, different protocols are used in different parts of the system, and so on, and so forth. It’s also crucial to understand the key responsibilities of SRE in ensuring the system’s reliability and scalability. Study the system as closely as possible, to the tiniest detail.

Your system might generate and be loaded, but these two processes should be regarded as separate while testing. You will emulate the source of the load with your loading scripts. Depending on the system architecture and the customer’s needs, you can offer your customer isolated or integrated tests, or both. Isolated tests check how well each system component works, provided everything works fine. After isolated tests, all the systems are connected back together, and in an integrated test, you look for bottlenecks in connections between the systems.

Integrated tests are usually easier to run since you only have to emulate users’ activity while all the connections between the system’s components are already there. If isolated tests give you data on the maximal performance of each system component, and then you can tune each separately, integrated tests help to determine how they perform as a whole.

7. Create a Load Testing Profile

One trait that unites many of our customers is that they believe they know their systems. The thing is, most often, they don’t. If the customer is making up the load testing profile, it may include the operations that the customer thinks are there, but they are not if you check the statistics. Customers often think their profiles are much more intense than they really are, overloaded.

You can write a profile and base it on data, not thoughts and beliefs, but it’s not a quick thing to do. If the time for that were planned according to the contract, you would have to get access to statistics and waste even more time when discussing it with the customer. Evaluate if you gave time and other resources for all those tasks.

8. Assess Labor Costs and the Team’s Competency

At this stage, you have access to the system and have looked at it. You know your QA team, and you know your timelines. Even if you don’t have a profile yet, try recording traffic: it’s something to at least start assessing labor costs. Even with the first recordings, you can evaluate how long a certain script would take.

As a performance testing manager, you are familiar with the team, so ask them what they are good at, how long they have been in QA, and their strengths and weaknesses. So you can either suggest your evaluation or ask team members to evaluate what they can do in how much time. The second approach seems more motivating since it suggests agency and self-imposed responsibility. Moreover, in every project, your team and your labor assessment skills will also build up.

Don’t forget to add time for debugging. The risks are high: bus factor, delays on the customer’s side, COVID, and other unforeseeable circumstances all ask for some extra time. Usually, the time you need can be calculated as a certain percentage of the number of scripts you plan to use. For three scripts, it may be 5%, while for a bigger number of scripts, you should add 10 to 20% of the time for debugging.

9. Plan with MS Project. Estimate Total Project Duration. Revise

Make a project plan using Microsoft Project or a similar tool. Evaluate each script, each stub, and staff member and make a plan using this or any other tool. MS Project is not the only tool you can use, but it has certain valuable features. For instance, your customers may see which tasks they are responsible for and how much of each task is completed by a certain date.

MS project helps you visualize how well your forecasts are working so that you can keep yourself and your customers informed. It makes revision easy and saves time. If you’re not able to meet the deadline, the earlier you see it and inform the customer, the more ways to compensate for a delay there are.

10. Use Trello, Backlogs, and Confluence

Transfer tasks from the plan to Trello/Jira or a similar task tracking tool. Gaining access to the system, deploying monitoring, and writing scripts – these are all tasks of different priority, and according to priority, they should be included in the backlog.

Created tasks have names, descriptions, and tags. You can also assign a specific participant if you think it’s important that this particular person should be responsible for it. You can add a checklist, a deadline, or write comments on the task. It is advisable to do so, especially if you move the task to the “Waiting” column, specifying the reason in the comment. When the task is completed, it is moved to “Done”.

Also, start making a project in Confluence. Create a project and fill in all the information. It is necessary so that if the customer returns in half a year, and the people who worked on this project will be unavailable, you will be able to retrieve all necessary information from Confluence. When creating it, add Performance testing methods and the final report. The more information you contribute, the better for subsequent projects.

11. Distribute and Manage Tasks in Performance Project

A lot depends on your deadlines. If your plan is realistic in terms of deadlines, you can use agile task management, and your staff will pick tasks themselves. Otherwise, you can assign tasks by yourself to develop the versatile skills of your team members. It’s safer to assign tasks if you’re in a hurry. No one knows your team’s strengths and weaknesses better than you.

Your tasks should be smart: specific, measurable, achievable, relevant and time-bound. “Write a message to the customer with a request for statistics” is wrong. Writing it will take a few minutes, while getting the statistics will take much more time. It is crucially important to set a specific task with specific deadlines.

12. Control Your Software Testing Team

It makes sense to hold daily meetings in the beginning of the day to discuss problems they faced yesterday and their plans for the upcoming day. Weekly meetings are mandatory. You should tell the team about the status of the project, remind them of the deadlines, revise the tasks, and adjust the plan. If you expect delays, it’s better to report them to the customer at an early stage.

Also, if some of your employees come without experience, it is better to check some of their tasks after them. If you do not check the script at the beginning of the project, you may end up with tests that don’t work. (See our article “Organizing the Best Software Testing Team“)

13. Communicate with the Customer on a Regular Basis

You should regularly report to the customer what has been done and how the project is proceeding (Check our “Software Testing Documentation Tips“). Here are some of our favourite ways to do so:

• Write down all arrangements and send meeting protocols to the customer by email.
• Discuss deadlines and every change in them. Write them down, too.
• Request for information from the customer. Include deadlines.
• If the customer delays providing information – repeatedly remind them of their obligations.
• Try to obtain written approvals from the customer. Email messages are preferable, since in messengers one can delete or edit messages.
• Escalate. if people responsible for communication do not perform, inform their managers.

14. Involve Developers and Analysts at an Early Stage. Report First Test Results

When you test other company’s systems, a lot of their staff will not trust you. Your test results show their system doesn’t work well? But for them, it might be good enough, and they may start doubting your competences. To avoid criticism at the end of the project, start involving developers and analysts from their side at the earliest stages possible.

15. Create a Performance Test Registry

First, create a test registry, an Excel spreadsheet. There, you will be able to write down dates, details, results, short summaries of changes made to the script, and conclusions. Write down that the response times are not higher than one second, or that the maximum performance is this and that, or that the problem was reproduced at such and such a moment of the test.

Lifehack: send yourself an email with results of every test, and then when you need to write a final report, and you do not remember anything, collect all those emails into one report, add conclusions, and that’s it. Final report will not take you long.

Conclusions. QA Manager Roles and Responsibilities in Fixed Price Projects

To sum up, you are responsible for the technical side of the project. Responsibility for contracts and orders, conflict resolution with the customer, communications with the customer, finance, and the non-technical results of the project in general are on the overall Project Manager. The PM is supposed to resolve any conflicts between you and the customer, and must participate in all communications with the customer. We will take a look at such problematic fields as budgeting, contract control, risk management and team selection in the next articles. Stay tuned!

Creation of high quality solutions is impossible without testing them, so an expertise in performance testing management is something very valuable in today’s world. As a company with 450+ professional engineers, we do know that. In PFLB, we have been providing performance and load testing services since 2008, and have done it for 300+ happy customers, so don’t hesitate to contact us, if we can be of any help.