How to Test a Release on Real iPhones and Android Phones Before Your Users Do
Test each release on real phones with a written checklist, one device at a time. Start from an old install that holds real data, upgrade it, then run every check. Every failure becomes a fix with a test, or a logged bug, before your users ever see it. The free AI starter kit we build our own products with runs these checks on real phones for you.
Write the checklist from the release
Read what changed in the release, across every repo it touches. Write one checklist in your docs folder, where each row says "do this, then you should see this". Keep it to what changed plus the flows that make money, like sign-up, the main task and paying.
Start with an upgrade
Your users don't install fresh. They upgrade an app full of their own data. So start on a device with an old version and realistic data, upgrade it, and run the checks. Do a fresh install last. If one device pairs with another, like a phone with a desktop app, test them together.
One device at a time
Take one device to a stable result before you pick up the next, and keep the same order every release. Check shared code once, on the fastest device, and save the slower devices for what's specific to them.
Report every check
Each row gets one of three results: pass, fail with what you saw, or not run and why. "Looks fine" isn't a result. A screenshot of each changed screen in the pull request shows reviewers what shipped.
Turn every failure into a fix or a logged bug
| What fails | What to do |
|---|---|
| A bug that blocks the release | Fix it on the release branch with a test that catches it |
| A bug that can wait | Log it for after the release, with the steps to see it |
| Data missing after the upgrade | Stop the release and check the migration |
| Works on one phone, breaks on another | Test the shared code on both before you ship |
Get the phone check we use
The starter kit plans the checklist from your release, runs your app on a computer, an iPhone and an Android phone, and tries every release before your users do. Get the starter kit free on GitHub.
Questions
Why test on real phones when the simulator works?
Real phones carry your users' data, upgrades, memory limits and network conditions, so bugs from those often only show up there.
Should I test a fresh install or an upgrade?
Both, upgrade first. Start from an old install with realistic data, upgrade it, run the checks, and do a fresh install last.
What do I do when a check fails right before release?
Treat it as a bug. Fix it on the release branch with a test if it must ship fixed, or log it for after the release if it can wait.