Brayden Haws

August 19, 2021

Metrics Framework

How do you know if the product you built is having an impact? Or if the project you delivered was successful? Or if you are truly meeting customer’s needs?
Delivery is often the focus of our work, the more we can get out into the world the better... or so we think. But where we should be putting more of our time is into measuring the impact of what we create. I’ve seen several frameworks for measuring engagement/impact/outcomes, with those in mind and drawing on my own experience, I’ve come up with the following metrics framework.
This approach is focused on what to measure at different points along the client/product journey. From when the product first lands in customer hands all the way until it is a piece of their daily life. And along the way making sure we are building something that is critical to our customers’ way of life, so it becomes something they cannot live without.
1. Get Started
  • It all begins with what I call Get Started. This is about giving yourself a reality check that you have something the customers want.
  • If you built an app, are people downloading it? If you sell a service, do you have buyers? This is all about measuring if you meet customers bare minimum requirements, do you have enough that they want to start the journey with you.
  • Other ways you might hear this phrased are things like: Table Stakes, Acceptance Criteria, MVP.

2. Short Term
  • In the Short Term you want to look at are customers using what you’ve built/delivered. And if they are using it, how are they using it and how often.
  • This is where you want to track things like Daily Active Users, Weekly Active Users, repeat usage (how many times a day is someone logging in), Usage of specific features or functionality.
  • You are focused on making using your product/service a habit for your customers.

3. Long Term
  • Now that you have met customers minimum needs and using your product has become a habit, you want to focus on the value it is bringing to them.
  • What types of ROI are customers seeing? Are they reaching the goals they had in mind when they first engaged with you? How do you you continue to deliver value to them?

Outside of these points in time, there are also things that you want to measure throughout the lifecycle of a product and its associated customer journey. These are things that make an impact all along the product journey.
Customer Voice
I have been doing a lot of reading on product leadership and great product companies. One big theme that comes up over and over again is that they are driven by data. They have countless data points for all the different timeframes above (and for anything else you can think of). However, what has been surprising to me has been how often these companies take huge risks not based on data, but based on the voice of the customer. 
A good example of this is Amazon Prime, financial models actually told Amazon that Prime was a terrible idea but since they were so close to their customers, they knew it was the right decision.
A sort of counter example from Toward’s Data Science is when Coke used millions of data points to drive the creation of Cherry Sprite. When they could have just talked to customers, who had been buying Cherry 7-Up for 30 years.
We should be constantly talking to customers, interviewing weekly (if not more). We should collect customer anecdotes and refer to them often. And as we make big decisions for the product or company we should do so with the voice of the customer in our heads.
All along the journey, as we measure: Get Started, Short Term and Long Term, we should look for the metrics or actions that make our products sticky. What are the things that make us essential to our customers and turn us from another product to something that is habitual? 
The classic example of this comes from Facebook, where once someone has 50 friends Facebook no longer worries about them churning. It is not about hitting the friend request button 50 times but rather what those 50 pushes represent, that once someone has 50 friends they feel a sense of connection, they have a reason to keep coming back.
Just as important as determining what those stickiness metrics/actions are, is determining the timeframe they need to occur in. Teresa Torres refers to this as traction. Thinking back to the above example, it is important that people get 50 friends, but it is also important to know the timeframe in which they are more likely to become sticky. Meaning are there better outcomes if they get those friends in 7 days instead of in 90?
What you track at each stage will vary based on the product you build or the service you offer. What matters is taking the time to determine what to measure at each step of that journey and then using this data to ensure that both you and your customers are successful all along the way.

About Brayden Haws

Healthcare guy turned tech wannabe. Doing product stuff at Grow. Building Utah Product Guild⚒️. Constantly tinkering on my 🛻. Occasionally writing poor takes on product strategy and technology⬇️.

Website | LinkedIn