A/B testing for mobile apps

A subscription app makes money in two places. One is inside the app, on the paywall. The other is everything outside it: the website, the web pricing page, a web checkout if you have one, and the ads that bring people in. Both are worth testing, but they need different tools, because you can change a web page in minutes and a native screen only through an app release.

So test the in-app paywall with a paywall tool, test the web side on the web, and judge everything on revenue rather than installs.

Why testing is different for mobile apps

The purchase often happens somewhere you can't measure from your website. A visitor taps "Download on the App Store", leaves your site, installs, and maybe subscribes a week later. Tying that payment back to the page they saw is hard. A web checkout keeps the purchase on a page you control, which makes it the cleanest thing to test.

The web side matters more than it used to. Since May 1, 2025, Apple's App Review Guidelines have let apps on the United States storefront include buttons and external links to buy outside the app (Apple Developer news). In December 2025 the Ninth Circuit ruled that Apple may charge a commission on those linked-out purchases limited to its genuine costs, with the amount left to the district court (Fenwick summary). The rules may change again, but for many US apps the web checkout is now a real sales channel.

In-app tests need an in-app tool. RevenueCat Experiments, for example, tests prices, trial offers and paywall designs inside the app (RevenueCat docs). See the paywall A/B testing guide for that side.

The tests to run first

These are for the web side. Each is judged on revenue per visitor, with installs and taps shown only as context.

1. Web checkout versus App Store link

On your website, send half of visitors who click "Get started" to a web checkout and half to the App Store. Measure revenue per visitor. A card form may convert fewer people than a one-tap store purchase, and the web checkout may still earn more per visitor. Only the revenue tells you.

2. Annual plan first on the web pricing page

Default the toggle to annual and show it as a monthly price. Measure revenue per visitor and the annual share. The pricing test guide covers the setup.

3. Trial versus no trial on the web

Compare a 7-day trial against paying on day one, for web buyers only. Measure revenue per visitor, counted after the last trial in the test ends. More in free trial vs no trial.

4. A short quiz before the web paywall

Some subscription apps ask a few questions on the web before showing the price. Test a 3-question quiz against going straight to the plan page. Measure revenue per visitor, since a quiz can lose people and still raise purchases among those who finish.

5. Ad copy that names one problem

In Meta ads, test copy built around one specific problem against your current feature-led copy, both sending people to the web checkout. Measure revenue per ad visitor. See Meta ads A/B testing.

6. A pause option for web subscribers

App Store subscribers cancel in their phone's settings, so you can't put a cancel flow in front of them. Web subscribers cancel through you. Offer a pause before the cancel button and measure revenue retained per customer who starts cancelling.

How much traffic you need

A made-up example. Your web pricing page gets 15,000 visitors a month, about 500 a day, and 5% of them buy.

To detect a 20% lift at 95% confidence and 80% power, a standard calculation needs 8,158 visitors per version, 16,316 in total. At 500 a day that is about 33 days. Add your trial length on top if the test involves a trial, because the last visitors' payments arrive after their trial ends.

If your web page only gets 3,000 visitors a month, the same test takes over five months. At that traffic, test bigger changes (web checkout versus App Store, a new price) and skip small copy edits. The duration calculator runs this for your numbers, and the minimum detectable effect calculator shows the smallest change you can detect in a set number of weeks.

Common mistakes

• Judging a website test on App Store taps. A tap is a click, not a payment.
• Forgetting account linking. If someone pays on the web, the app must recognize them on first login. A broken handoff shows up as refunds and support emails, not in the test result.
• Calling a trial test at day 7 when the trial is 7 days long.
• Testing the in-app paywall and the web checkout at the same time on the same users, then crediting one for the other's lift.
• Treating US and non-US users as one group in a test that depends on linking out from the app, when the rules differ by storefront.

How Outtest fits

Outtest tests the parts of an app business that live on the web: the app's website, the web pricing page, the web checkout and Meta ad copy. It does not change the native app or the in-app paywall. It connects read-only to RevenueCat, which lets it read your App Store revenue, and to web payment tools such as Stripe or Paddle, then judges each test on revenue per visitor.

The Referee agent calls a winner only after at least 7 days, a 90% chance of beating the original and a lift of at least 10% (the defaults, adjustable in Settings), and calls a draw at day 42 if the bars aren't met. Checkout and pricing changes always wait for your approval. The install is one script on your website.