How do you decide what to measure?

It’s a question I keep circling around.

Sure, we have metrics other smart people told us to use, but how should we use them? How do I know they matter to me in my situation right now? How do I know when they’re not enough?

A game I’ve started playing is “Who did it better?”—but for website functionality.

You can do this for anything: compare two basketball players, two cell phones, two boxes of cereal. It’s best when you compare a good vs bad example. Functionally, they do the same thing, but one is clearly better than the other. How? What makes that one better and the other worse? These are your criteria, your signs of quality. Measure them.

If both websites load, which loads better? If both websites search, which searches better? If both websites deliver custom flower arrangements, which one does it better?

The trick is talking about real examples out loud. As soon as you pit two real examples against each other, you intuitively construct a hierarchy of metrics—things you can use to rank a site’s performance. Talking about it out loud helps expose your priorities, and talking with someone else forces you to explain your thinking, to articulate which qualities are most important to you.

Use the “Who did it better?” game to help validate your metrics.

If your metrics aren’t telling you whether you’re getting better, define new ones that will.