Case Study: RunKeeper Drives User Engagement With A/B Testing

We are thrilled to invite you to our webinar on November 14th at 2 PM EST with RunKeeper's Product Manager Jon Gilman and Localytics' VP of Marketing Bernd Leger. Sign up here to join this webinar and learn how to use data to drive user experience and engagement for your app.

In the meantime, hear how RunKeeper uses Localytics in this case study:



RunKeeper logo

RunKeeper is the leading health and fitness app for iOS and Android with more than 25,000,000 organically-acquired users. Like a “personal trainer in your pocket”, RunKeeper uses the sensors built into phones to track activities such as running, walking and biking. For these activities users can measure their pace, distance, calories burned and more. While doing this, the app also provides audio coaching, plans, and connects users to a community of other users working towards the same goals. RunKeeper’s superior user interface, distance tracking accuracy and loyal user community make them the premier running app, and a standout in a competitive market of health and fitness mobile tools.



RunKeeper’s goal is to maintain and advance its category leadership in running. In addition to owning the running category, RunKeeper’s bold company vision is to create the best way for people to stay fit and to expand beyond running, aiming to help users track all of their health and fitness activities.

To achieve this, the team re-imagined their app’s design and thought about ways to ignite enthusiasm and awareness for using the app to track activities like weight lifting, yoga or biking - not just running. RunKeeper has an incredibly agile product team that ships app updates every couple of weeks. They needed an analytics solution that could keep pace with their design changes and provide them with immediate insight on how these changes impacted user engagement and usage of key features. In particular, RunKeeper needed user-level data to create segments of users based on activities logged in the app. They hoped to compare the engagement of these segments in order to identify what events drove the highest engagement, and then design the app around those features.

Prior to using Localytics, RunKeeper used an internal analytics dashboard for download and app usage tracking. However, the process of wading through spreadsheets of raw data, gleaning actionable insights from those metrics and creating understandable reports to share with the company was time-consuming and daunting.

During the evaluation of mobile app analytics platforms, Localytics stood out from the competition for because of its user-level analysis and ability to provide those much-needed segments. Localytics eliminated hours of analyzing spreadsheets and empowered the team to achieve the iterative and agile approach to analysis, product planning and decision making that they needed.

“Localytics saves me time and makes it a lot easier to communicate test results. It provides actual insight. Localytics makes my life easier,” says Jon Gillman, Product Manager at RunKeeper.


Using Localytics

One of the first major projects RunKeeper used Localytics for was the analysis of a design A/B test of their start screen. The goal of the test was to see if a design change provoked users to log activities other than running. Here's a look at what they tested:

Localytics cae study RunKeeper mobile app AB Test

“Changing the start screen was a high risk move but having Localytics gave me the confidence to test this,” says Gilman.



RunKeeper used Localytics’ funnel analysis tools to split the reports with custom dimensions for the A/B test groups and analyze which UI variation resulted in more non-running events logged.

In the A/B test’s winning version, RunKeeper saw a 235% increase in non-running activities logged per user in the winning. Overall, the winning variation was 10 times more successful at driving users to log a non-running event than the other version. In addition, since the A/B test was exclusively done in the Android app, RunKeeper can port the winning variation over to the iPhone app with extra confidence – saving development time on iterations later on.


What’s Next?

Looking ahead, RunKeeper will continue to use data to test and evolve the app experience around a broader set of activities.

“Just the fact that people are tracking yoga makes us think about how we can do a better job of driving insights around these activity types and help people track progress around these kinds of activities in the app,” says Gilman. “For example, perhaps we can help users track Vinyasa sessions versus other kinds of yoga.”

RunKeeper will quickly move toward fulfilling their company vision of helping people become more fit and healthy – no matter what fitness activity their users track. To do this, they will continue to evolve the app at a rapid pace. Looking ahead to the future, they plan to use the Localytics Marketing in-app messaging feature to encourage users to upgrade their apps, helping users enjoy the latest features and live healthier lives using RunKeeper.


runkeeper webinar cta