System failures and website crashes have been an issue as long as businesses have had websites and/or applications. Some notorious examples include eBay’s crash in 1999 with $4 million losses, or FOREO’s website crash on the Black Friday with about $10 million of lost revenue. Heck, weeks ago Zuckerberg lost $6 billion in a few hours after the Facebook outage.
In PFLB, we strongly believe in the importance of performance testing for business. It’s sensible, more rational and more effective to run performance tests and avoid the crash rather than mend the system after the crash, losing not only revenues, but also man hours. In this article, we’re proving the point: we have been providing performance testing services since 2008, and have done it for 300+ customers, so our 400 employees do know a little something about how performance testing can change your business.
Request a quote, if you want to discuss how your business can benefit from performance testing services.
How much does a system failure cost?
So in 1999, eBay went down for 22 hours. The company estimated that they had lost around $4 million, or almost 2% of their annual income. But even if you are not eBay, just google system failure or website crash: you will be surprised by how often it happens to banks, on-line shops, governmental websites, etc. Such giants as Facebook or Instagram are not safe either, besides a huge number of IT specialists on staff.
Meanwhile, the consequences for business are not something you can pretend are nonexistent. They include:
Let us give you some examples. One of the most common cases of lost profits is a website crash under peak load on Black Friday. Seriously, many of our clients come in the end of November, although nothing kept them from running tests before the crash, not after it. But for them, sales just don’t happen, because the moment has gone.
After a shutdown, you’ll have to put everything back on track, which is also costly, especially if you have to rebuild or at least repair the infrastructure. Headlines in the media that mention the crash lead to reputational losses. These, in turn, can lead to reduction in your market value, as it has just happened with Facebook. Moreover, if you’re in e-commerce and in Google ratings, you can slip down the rating list. As for productivity issues, a number of your employees won’t be able to work while the system is down, and this is money in salaries, lost for you, too.
Some of the biggest companies have already developed their own approaches to calculate how much system changes cost them. Take a look at a few examples:
How much can your business loose?
But how do they calculate it? Well, it is obviously proprietary information, but in fact there is quite a universal formula that lets you calculate your own, hopefully potential, losses. Roughly. Using it, you can estimate your losses and see just why performance testing is so vital for your business goals.
1. For lost profits, divide your amount of annual sales by 525600, that is, the number of minutes in a year, and multiply the number by the number of minutes that your website or app was down. That is how much you lost or can lose, in sales.
2. For costs to fix the system, multiply the average salary p/h in USD by the number of people involved, and then by the number of hours they (will) spend putting the system back on track.
3. For productivity issues, multiply the average idle employees’ salary p/h in USD by the number of idle employees, and then by the number of hours they had to stay idle
4. Next value is projected revenue decline due to loss of customer loyalty. To calculate it, multiply value from step 1, lost profits, by the percentage of repeat sales. Alas, you won’t know about repeat sales until after the accident and a while more, which is why this value is called projected.
5. The last indicator is the projected decrease in revenue due to reputational losses. It is calculated in a similar way: lost income is multiplied by the percentage of sales to customers who came to the website from social media to compare goods and/or services.
You can look at your company’s history and calculate the number based on the shutdowns you have already had, or try to calculate it for a potential shutdown. You will anyway need it to compare to the cost of performance testing to see if it benefits your business or not.
What is performance testing?
Performance testing makes it possible to assess the speed of operations, ascertain the permissible number of simultaneously active users or simultaneous transactions, and determine how the volume of data affects the performance of an IT solution. The English of this is that performance testing is creating a load by emulating real users’ actions with software means to research how the system performs under load.
Performance testing is just right for you, if you need to:
In each of these situations, performance testing helps your business achieve its goals. With its help, you can:
But the main thing is that regular performance testing allows you to take control of the situation and guarantee high quality of services.
What is NOT performance testing?
So, performance testing sounds like a magic wand, doesn’t it? Why are there still companies that don’t use it, then? The thing is, many businesses think they do use performance testing, but in fact they don’t.
There are a number of useful tools that people may mean when they say they load test their products or systems regularly, but, although useful, these instruments do not equal full circle performance testing. These are NOT a substitution for performance testing.
Mission-critical moments
In every healthy business environment, there are pinpoints, at which performance testing is just inevitable. They are, as follows:
If your business history includes or is supposed to soon experience any of these stages, consider looking for a performance testing services provider. And keep it mind that the perfect time to do so is 4 to 6 months prior to the event. Nobody can help you avoid the losses 10 days before the Black Friday. Fix the roof before it starts to rain.
Approaches to performance testing
There are two opposite approaches to performance testing: you can do it yourself or outsource to a performance testing company. Both have their ups and downs. If you make a decision to do it yourself, consider the following challenges:
If you, on the other hand, decide to outsource your performance testing project, you still need to be ready. The performance testing specialists will need data, access to your DBs, and your stand.
There is always an option to combine the two approaches, though. If you don’t want to pay for performance testing every time (and there will be many times, as discussed below), you might want the performance testing company to share their expertise. There are many ways to do so, but they should be discussed before you start the project.
Performance testing frequency
To cut a long story short, performance testing should be a part of DevOps. You run it once, you see the results and the benefits, but then the next release of your software is ready, and then what? Performance testing results go out of date very quickly, so you have to repeat it regularly in order to avoid system shutdowns and crashes. One way to do so is to integrate it into DevOps.
Stages of performance testing life cycle
Beware: your first experience with performance testing will most probably be just a sneak peek into the future of your business. It’s not like things don’t work the first time: performance testing does, but soon enough you’ll find yourself moving in the direction of regular tests. You will (or should, anyway) pick out engineers who will be responsible for the tests, find resources, and eventually, run tests before every release or launch. There are many useful tools that help automate this process: have a look at our articles on Top10 best load testing tools or Best online load testing tools.
But that’s not it, either. Your ultimate goal is shift left testing. There, you start testing along with designing your technical assignment. Your specialists test new code, go back to their colleagues from development and give them feedback on any risks of failures, if any. Then, they can proceed to the next stage or keep working with that same part of code. Similarly, you test every build and every deployment.
A prominent feature of shift left testing is that it is both cheaper than testing the final product, and it decreases your product’s time to market. You will never again have to put a fully developed product back to the development stage to start from scratch. Instead, due to timely testing, there will be no surprises before the launch.
Performance testing: stages of life cycle
In an ideal environment, performance testing starts along with the design of technical assignment and the first lines of code.
Conclusions
A perfect performance testing solution should benefit several groups of staff members. It has to save time and be convenient for developers and QA specialists. It is supposed to help management ensure transparency of the department’s work and save time on building regular reports. It obviously should save money for business, one way or another. Don’t settle for anything less.
At PFLB, we’re happy to share. Don’t hesitate to contact us, if you have a project in mind. We’ll help you find an ideal solution for your company.
Related insights in blog articles
Roles and Responsibilities of Performance Tester
The core of efficient performance testing is an experienced, certified and well-structured team of professionals, who have already worked in similar projects, and learned all the peculiarities of QA testing types, and protocols. If consciously chosen, they can evaluate a product, test its behavior under a load, check its response time, etc., and thus, empower […]
Tips for Efficient Web Application Performance Testing
Performance testing is one of the most challenging components of web application testing. But it is worth it: if done right, performance testing can forecast product behavior and its response to user’s actions with an impressive amount of detail and precision. Irrelevant web performance test data or wrong metrics chosen during testing cost companies a lot of money and effort, while not monitoring your web application performance at all can lead directly to a crash, say, on Black Friday, if you are not prepared to handle an increased number of concurrent users. Business or product owners needn’t know exactly how to performance test a website, but it’s useful to have a basic understanding of procedures that you pay for. From this post, you can learn what web application performance testing is, when it should be done, and how. Then, you will be able to make an informed choice whether to hire testing specialists or run testing sessions on your own.
Top Tools for Developers in 2024
The ratings of the best software development tools are time-sensitive: new tools emerge every now and then. Monitoring and mastering them is a full-time job, and sometimes a waste of time, too, since some oldies but goldies remain chosen by the team. Having done all the hard choices for you, we are sharing our list […]
Big Data Europe 2024
Big Data Conference Europe is a four-day conference with technical talks in the fields of AI, Cloud and Data. The conference will take place both on-site and online, providing the opportunity for everyone to participate in their preferred format.
Be first to know
Once a month we’ll be sending you a letter with all the useful insights that we could find and analise
People love to read
Explore the most popular articles we’ve written so far
- Android vs iOS App Performance Testing: What’s the Difference? Dec 9, 2022
- How to Save Money on Performance Testing? Dec 5, 2022
- Cloud-based Application Testing: Features & Types Apr 15, 2020