At a glance
- Client: a biotechnology company building DNA research technology, whose inventions include a DNA sequencing system; MIT ranked it third in its list of the 50 smartest companies of 2016
- System under test: a laboratory information management system (LIMS) automating information management for genetic blood analysis, from application filling through genetic material analysis to the moment a doctor receives the results
- The combinatorics: a rack under test holds 96 test tubes (4 of them control samples), and every tube is examined by 30 variants of one test
- The bottleneck: manual test scenario preparation took up to 4 hours per test tube, because the test data had to satisfy special conditions during combination
- Regulatory context: the assembly line analysis had to be tested in accordance with United States Department of Health and Human Services requirements
- What was automated: test data creation, and testing of the genetic blood analysis itself, using Selenium WebDriver, Java for writing the autotests and C# for running them
- Result: test data preparation dropped from 4 hours to 2, test coverage rose by a factor of 12, and overall testing performance rose by a factor of 40
- Cadence: the client deploys updates every 2–3 weeks with regular patches in between; the autotests now run every night
- Compliance dividend: the client used PFLB's automated testing reports as supporting documents for yearly certification by the US Department of Health and Human Services
Why a lab system cannot be tested casually
The client's priority was automatic examination of the LIMS that manages genetic blood analysis end to end - application filling, genetic material analysis, and delivery of results to a doctor. The assembly line nature of that analysis required meticulous testing, in accordance with United States Department of Health and Human Services requirements.
The arithmetic explains the problem better than any description of it. One rack holds 96 test tubes, four of which are controls. Each tube is examined by 30 variants of a single test. And preparing the test scenario for one tube manually took up to four hours, because the test data had to satisfy special conditions when combined - you cannot simply generate plausible values, they have to be internally consistent with each other. Multiply that out and the testing effort stops being a task and becomes a ceiling on how often the product can change.
Underneath the arithmetic sits the reason nobody was willing to just test less. The system must not be prone to errors, because any disinformation about a client's health can have serious consequences for that person and their relatives - and the company's reputation with it. This is a category of software where an undetected regression is not a support ticket.
What needed solving
- Automation of test data generation - removing the four-hour manual assembly per tube.
- Verification of the genetic analysis' life cycle - checking the process end to end, not just individual screens.
- Dependency examination for analysis parameters and results - confirming that inputs and outputs relate the way they are specified to.
- System's precision examination - confirming the results are right, not merely produced.
What was built
PFLB automated two things: test data creation, and testing for genetic blood analysis. The toolchain was deliberately plain - Selenium WebDriver as the automation testing tool, Java to write the autotests, and C# to run them.
Given how intensively the system is developed - updates every 2–3 weeks plus regular patches - the approach increased testing performance by a factor of 40. The autotests now run every night.
The automation caught real defects, in time to matter:
- Performance decrease after changes in system functionality and database structure.
- Errors in analysis results.
- Input analysis data that differed in the doctor's report - the most consequential class of bug in a system whose entire purpose is delivering a correct result to a clinician.
Results
- Test scenario preparation per tube: 4 hours → 2 hours.
- Test coverage: up by a factor of 12.
- Testing performance overall: up by a factor of 40.
- Autotests run nightly, against a codebase that changes every 2–3 weeks.
- A set of critical bugs at the level of genetic material research was eliminated.
- PFLB's automated testing reports served as supporting documents for the client's yearly US Department of Health and Human Services certification.
Questions this engagement answers
How do you automate test data when the data has to satisfy constraints?
You build a generator that understands the constraints rather than a script that fills fields. Here the test data had to satisfy special conditions during combination, which is exactly why the manual version took four hours a tube - and exactly why automating it halved the time while multiplying coverage twelvefold. That is the core of test data management.
What makes test automation pay for itself?
Change frequency. This client ships every 2–3 weeks with patches in between, so every release would otherwise have re-incurred the manual preparation cost. Automation is worth most where the same work repeats on a short cycle - and least where a system is tested once and frozen.
Does automated testing help with regulatory certification?
It did here. The client submitted PFLB's automated testing reports as supporting documents for annual certification by the US Department of Health and Human Services. Test evidence generated as a by-product of nightly runs is more defensible than evidence assembled for an audit, which is the practical argument for software validation built into the pipeline.
What kind of bugs does this catch that manual testing misses?
The ones that only appear across the whole life cycle: performance degradation after a database structure change, errors in analysis results, and input data that did not match what appeared in the doctor's report. None of those are visible in a single-screen check.
Testing a regulated system faster than you can prepare data for it?
If preparing one test scenario takes hours because the data has to be internally consistent, the constraint is not your test suite - it is the generator you do not have yet. That is buildable, and the same automation output doubles as certification evidence.


