Don’t measure your product using a north star metric
What is a “north star metric”?
Some people try and summarize their products with a single, all-encompassing metric usually called the “north star metric.” For example, Facebook is famous for using “monthly active users” and Pinterest for using “weekly active repinners and clickers.”
There are situations in which a north star metric is extremely useful – having a single metric allows for a clear aspirational goal to unify the entire company and celebrate success. For example, Facebook set a clear mission to reach 2 billion monthly active users – this number was advertised as their next milestone and was widely celebrated when achieved. Similarly, Airbnb celebrated 1 million nights booked, and Pinterest celebrated 235 million boards created.
But don’t be fooled – these metrics are only used to externally share the company's success. Internally, many different metrics are used to measure growth, and relying on just one key metric to inform growth can lead to myopic decision making and poor tradeoffs when deciding what to build.
To see problems that can emerge from using a single north star metric, let’s take a look at our friend John.
John is working on his podcast app. Using his app, users can subscribe to their favorite podcasts, share them with friends, and get notifications when there are new episodes available. John wants a way to measure how quickly his app is growing, and he decides to pick a north star metric.
John goes on Google, searches “good north star metrics”, and picks daily active users (DAU).
But this is bad. Why? Well, for starters, it’s horribly exposed to seasonality – by looking at daily active users, John is assuming users will use his app in the same way every single day. But when building a podcast app, this might not be the best assumption since users listen to podcasts most frequently on their commutes. On weekends, most people don’t have commutes, so the metric would be useless on Saturday and Sunday.
Okay, well what if John picked monthly active users? Well, in that case he has to wait until the end of each month to measure whether his product grew!
So John settles on weekly active users.
But that’s only half the problem, because John hasn’t even defined what “active” actually means! If I open the app, am I an active user? What if I’m not signed into an account? What if I accidentally tapped on the app, opened it, then immediately closed it? What if your app is running in the background every time I unlock my phone?
My point is this – every product is different, and using one metric just because another successful product used it doesn’t actually work, since how your users actually use your product might be completely different.
So John does a little more thinking and decides to pick a metric tied to the main action users perform on his app, and picks Weekly Episodes Viewed.
You probably know what I’m going to say – this is bad. For starters, it’s ambiguous – if it goes up a lot, does that mean users are viewing more episodes or are you just getting a lot more users? If you’re getting a lot more users, are those high-quality or low-quality users (because it matters if you’re spending a lot of money to acquire them!).
In addition, it’s too narrow – what if people start subscribing to more podcasts? What if people are starting more episodes but not finishing them?
The point of this blog post isn’t to just berate John’s decision making. To be fair, he is picking a frequency – weekly – that makes sense, since users generally listen to podcasts on their commutes. And the user action he’s measuring – episodes viewed – is probably the best indicator of engagement with the product. If users are watching more podcasts, they’re not only discovering new content that will keep them engaged but also getting more value out of the product as a whole.
So what’s the problem? This north star metric is still too narrow, and doesn’t give John or anyone else a good sense of how the product is growing.
## Of course, John could continue creating other, more granular metrics.
He could combine both Weekly Active Users (WAU) and Weekly Episodes Viewed into Episodes Viewed/WAU, which normalizes the metric and incorporates both user growth and engagement growth. But then what if WAU goes down, making Episodes Viewed/WAU go up – clearly this is a bad outcome, but he wouldn’t immediately detect it.
Or he could look at Revenue/Weekly Active User, which captures the value generated for the business – but this doesn’t represent customer engagement. If you suddenly doubled the number of ads shown to users, you’d increase Revenue/DAU significantly, but users would quickly stop using your product as much and in a month or two you might be left with half as many users watching half as many podcasts. Not a good outcome.
Okay, okay, we get it. So what *should* John (and everyone else) do?
Stop treating “north star metrics” like the end-all be-all metric and accept the fact that there is no one metric that summarizes your entire product. Products and users are complex and can’t be summed up by a single “north star” metric.
What would happen if tomorrow, Apple or Facebook decided to stop reporting anything in their quarterly earnings report except one number? Well, aside from potential SEC lawsuits, the stock price would tumble because nobody would have a clue how they were growing!
In the same way, your products growth is not a linear function that goes up and to the right. Understanding your product’s growth is like reading a story, not a graph. What you can do, however, is come up with a list of “guidance metrics” that are informative and actionable.
Let’s help John with this.
Create a list of core user actions – what are the three main actions users can do when using your product? For John, this list is:
Viewing a new episode.
Subscribing to a new podcast.
Uploading a new episode for a user’s own podcast.
Find a good metric to measure how many users you have. Say Weekly Active Users (WAU), where “active” is defined by someone opening his app for at least 5 seconds.
Now, put (1) and (2) together! John’s guidance metrics are:
Episodes Viewed/WAU
Subscribes/WAU
Uploads/WAU.
Now, John can either create a growth model using these metrics to forecast his product’s growth, create dashboards of these metrics over time so everybody can understand how the product is growing in realtime, measure the success of new features based on what user behavior they should drive – the possibilities are endless!
By using the three steps outlined above to come up with guidance metrics, you can ensure you’re constantly measuring your products growth without being overly focused on one north star metric.