At a glance
- Client: X5 Retail, a food retail company worth $16 billion, operating thousands of grocery and other stores
- How it started: X5's single load testing engineer left, weeks before high season - the period when retail systems carry their heaviest load of the year
- What we inherited: outdated documentation, an old performance testing methodology, production incidents, and data centre hardware already running at its limit
- Scope: the SAP ERP business processes retail actually runs on - updating price tags across outlets, daily discount calculation, daily delivery logistics - driven by data every store sends to the central unit each day
- Simulation: production load from thousands of retail stores plus thousands of internal SAP users and the heavy background processes running alongside them
- Duration: more than 10 years, from SAP ERP platform deployment through to an established testing process
- Built for the project: Express Report, PFLB's own monitoring system, because the client could not use Grafana at the time
The bus factor, arriving at the worst possible moment
For a while X5's SAP Basis team had the capacity to watch performance themselves. Then the network grew, the systems got more complex, the load rose - and so did the team's workload. So the retailer moved load testing to one of its own full-time testers, which held for a couple of years.
Then that engineer left. The replacement had to be found immediately, and it was just before autumn - the start of high season, when a retailer's systems process the flows that keep thousands of stores trading.
PFLB stepped in with specialists straight away. That is the unglamorous part of the story and also the point of it: the risk was not a slow report, it was a national grocery chain going into its busiest quarter with nobody watching performance.
What "rebuild the process" actually meant
The handover materials were a pile of documents years out of date and a performance testing methodology to match. The process had to be reconstructed from scratch - collecting current information piece by piece and putting the documentation flow back in order - while simultaneously dealing with production incidents and with flaws in the testing process that only appear under high-season conditions.
The hardware made it harder: data centre equipment had not been updated in a long time and was working at the limit of its capability, while the chain kept expanding.
What gets tested in retail SAP ERP
The processes that matter are not the obvious ones:
- Updating price tags at retail outlets
- Daily discount calculation
- Daily delivery logistics
All of them driven by what every store reports centrally each day - stock supplies, balance forecasts, expiry dates.
The test environment had to approximate production closely: thousands of stores' worth of load, thousands of simultaneous internal SAP users, and multiple heavy background processes. Some processes are light but business-critical, and the discipline is not to let those fall out of the load profile just because they generate little load.
Two constraints ran throughout: new processes appeared and old ones changed constantly, so the load profile had to be adjusted quickly - faster still when a production issue needed urgent testing - and the test environment was shared with functional testing, automation teams and SAP consultants, which makes scheduling part of the engineering.
What we built and left behind
- Updated performance testing methods and current instructions, replacing what had gone stale.
- Detailed configuration checklists for the test environment covering each recurring situation: after copying data from backup, generating test data, preparing for load testing, and stopping tests and collecting results.
- Express Report - PFLB's own monitoring system - to collect and compare logs, with tooling to compare metrics for the main multi-threaded business processes and their child tasks, since Grafana was not available to the client then.
- AWR report analysis, verification of test data generation accuracy, and tracing for calculation-heavy tasks, extracting child tasks and analysing them at runtime.
- Automation of all of the above, progressively, as the project matured.
Results
An agile performance testing process was built for one of the largest retail companies - and built during a high-season crisis, concurrently with a hardware upgrade, rather than in a quiet period. The relationship has run for more than ten years.
Questions this engagement answers
What happens when the only person who knows your load testing leaves?
This is that case. The answer was an immediate specialist handover followed by a full process rebuild - because inherited documentation years out of date is not a starting point, it is a liability.
What does SAP ERP performance testing cover in retail?
The daily operational processes: price tag updates across outlets, discount calculation, delivery logistics - simulated against load from thousands of stores and thousands of internal SAP users.
Can performance testing be set up during peak season?
It was here, alongside live production incidents and a hardware upgrade. Not the recommended sequence, but a demonstrated one.
What if the client's monitoring stack cannot support the testing?
You supply it. Express Report was built to collect and compare logs and metrics across multi-threaded business processes when Grafana was not an option.
Running retail on SAP ERP?
High season does not negotiate, and the processes that break are rarely the ones being watched. The useful work happens in the quarter before.
