QA Costs Down Almost 30% for a Bluetooth Fitness App — 300 Manual Tests a Day and a Rented IPv6 Tunnel

Volha Shchayuk

Author

Volha Shchayuk

Volha is a seasoned IT researcher and copywriter, passionate about AI, QA, and testing. She turns technicalities into engaging articles, helping you discover and easily grasp the latest IT concepts and trends.

IT researcher

Reviewed by Boris Seleznev

Reviewed by

Boris Seleznev

Boris Seleznev is a seasoned performance engineer with over 10 years of experience in the field. Throughout his career, he has successfully delivered more than 200 load testing projects, both as an engineer and in managerial roles. Currently, Boris serves as the Professional Services Director at PFLB, where he leads a team of 150 skilled performance engineers.

Growing popularity is a testing problem

The client came to PFLB at the point where their popularity grew and the product stopped standing still. New functions were being added, additional features were landing, new workout games were being developed - all while the existing audience was expected to keep using it every day.

That is a particular kind of pressure. For a fitness product, quality and usability are retention: if the app stutters during a five-minute workout or fails to pair with the device, the user does not file a bug, they stop exercising. The requirement the developers set was blunt - the application had to be user-friendly, intuitive, and work flawlessly anytime and anywhere on the globe.

Two goals framed the engagement. As the application's functionality grew, developers needed to broaden regular functionality testing within a limited amount of time. And they needed to cut the cost of testing by improving the efficiency of the QA work - not by testing less.

Starting manual, on purpose

The project began with manual testing, as the quickest way to establish quality on a product that was still moving. The daily rhythm was measurable rather than impressionistic:

• 20 different devices in the test pool.
• 300 manual tests per day across Android and iOS for each regression.
• 150 tests per day for smoke testing.

The supporting toolchain was deliberately unglamorous: Charles Proxy for API testing, Xcode and Android Studio for recording all logs, and test builds distributed through TestFlight, Crashlytics and Google Alpha.

Automating iOS - and why Appium won

Alongside functional testing, the team implemented test automation for the iOS apps. Two options were on the table: Appium and XCUITest. Appium - a test framework built on Selenium and JsonWireProtocol - was chosen, and it was chosen for cost.

The decisive argument was forward-looking. With Appium there is no need to rewrite the tests for Android later; extending coverage means adding new locators and new Appium server settings, not building a second suite. On a product where the roadmap already implied Android automation, that removed a duplicated project from the future.

Testing an app that talks to a device

One key feature of this project was that the app is not self-contained. It is the companion to a portable fitness device connected over Bluetooth, so the interaction between app and device had to be tested directly - not simulated. On top of the usual functional testing and test automation, the scope had to include Bluetooth compatibility across mobile phones and stress tests, using specific BLE tooling.

The team's practical note from this work: integrating the app with Crashlytics is the fastest route to knowing whether a build is stable in the field, because it gives continuous access to analytics rather than waiting for a report.

Two obstacles worth naming

IPv6. The initial submission to the App Store surfaced a review-guideline requirement that the app must work on IPv6-only networks. Since IPv6 is only about 1% of network traffic, getting decent IPv6 coverage in an office is not a configuration change. Against a very tight deadline, the team rented a VPN server in the United States to tunnel an IPv6 network into the office - which is what allowed all pre-release testing to finish on time.

Three teams, three continents. The Android, iOS and game applications were developed by three separate teams located on three different continents. Linking them up was a genuine challenge at the start of the project. Working under Agile, the QA team was integrated into the developers' regular sprints - and then the time zones were turned from a liability into the schedule itself: testing began right when the developers' workday ended, so that when the client returned the next morning, all tests had been executed and all bugs submitted to the tracker. That single arrangement cut release time significantly.

Results

• QA costs reduced by almost 30%.
• Time-to-market significantly improved, thanks to the 24-hour work shifts created by the time-zone split.
• The app itself improved - gaining popularity and receiving more positive reviews on the App Store and Google Play.
• Beyond functional and regression testing and regular test-model optimisation, the team constantly analysed user reports, which is a large part of why the review trend moved.
• All pre-release App Store requirements were met on schedule, including IPv6-only network behaviour.

The bug tracker the engagement ran on: open defects by severity and priority, with the burndown across the release.

A single defect as reported: reproduction steps, environment, severity and attachments - the detail a developer can act on without a follow-up call.

The success factors the team credits are process ones, not tooling ones: constant clear written test processes; a project manager close to the team so response time never exceeded one hour; testers available for urgent work, especially around release deadlines; a single tool for test management and bug tracking; and Agile throughout.

Questions this engagement answers

What does manual testing actually cover on an early-stage mobile product?

Here, 300 tests a day across Android and iOS for each regression, 150 a day for smoke, on 20 physical devices - plus API checks through a proxy, log capture on both platforms, and continuous review of real user reports.

When should a growing app start automating?

After manual testing has established a stable baseline, and starting with the platform where the cost case is clearest. This team began manual because it was the quickest route to quality, then automated iOS with a framework that would not have to be thrown away when Android followed.

Appium or XCUITest?

Here, Appium - because the Android suite would otherwise have to be written from scratch later, and because it lowered the cost of automation for the client. XCUITest is the stronger native choice; it is not the cheaper cross-platform one.

Can QA costs actually fall while coverage grows?

They did here - almost 30% down, with functionality expanding throughout. The saving came from efficiency of QA work and 24-hour shifts, not from reducing what was tested.