Maintaining app ratings

If you've already got a significant user base on your app, you are likely pressured to maintain a 4+ rating on the app stores. App ratings determine the likelihood that your app may be featured prominently in listings and search results, which has a natural and direct correlation to your ability to expand your user base organically.

For instance, at a company like Intuit, despite the fact that there are a number of experiments that run during tax season, the pressure to keep ratings above 4.0-download rates decreases significantly as ratings drop:

Running experiments, which are necessary to evolve your app, can have uncertain impacts on the ratings.

Healthy ratings and ongoing experimentation are both necessary when applying the Lean approach to app development. However, since reduced ratings negatively impact user acquisition, you will need to find a way to minimize such impacts. Finding a balance between potentially disruptive experimentation and good ratings is an ongoing challenge, especially as an app matures.

In the early stages, moving fast and breaking things works well. However, for an established app with healthy ratings and a healthy user base, it can be difficult to rationalize experiments that could cut ratings. After all, lower ratings decreases the likelihood that your app will be featured or receive an editorial listing, both of which can massively increase exposure and downloads.

Justifying experiments to other team leads may be challenging, but it can also be necessary, even after an app is well established. After all, the more an app becomes successful, the more likely it is to gain competition. Later, we will discuss ways to run split tests that provide useful data without causing too much disruption.