If you see yourself as a rational person, you probably try to retrospect occasionally and evaluate whether what you’re doing is right.
This is difficult for many reasons, among those what is “right”? Also, how do you avoid your own biases and look at things as they are, rather than making yourself feel good (or bad) unfairly?
The same extends to organizations, trying to make data-driven decisions and measure success objectively. “You can’t improve what you don’t measure” is the mantra.
It’s a noble pursuit generally, but often taken too far. Some of the most important things are impossible to measure. Doing good in the world. Empowering others. Living true to yourself. There are no numbers to track there.
You can track things, which you think are correlated with the thing you care about. However, you have to keep in mind three complications:
This is difficult for many reasons, among those what is “right”? Also, how do you avoid your own biases and look at things as they are, rather than making yourself feel good (or bad) unfairly?
The same extends to organizations, trying to make data-driven decisions and measure success objectively. “You can’t improve what you don’t measure” is the mantra.
It’s a noble pursuit generally, but often taken too far. Some of the most important things are impossible to measure. Doing good in the world. Empowering others. Living true to yourself. There are no numbers to track there.
You can track things, which you think are correlated with the thing you care about. However, you have to keep in mind three complications:
- you can’t be sure they’re correlated (the only way to be sure is to have data, but we’re already saying the important thing can’t be measured directly),
- even if they are correlated under some conditions, it’s likely not universal,
- there are probably multiple metrics correlated with your goal and it’s hard to know which to pick.
You have to choose based on gut feel, I.e. subjectively, however hard you try to be objective.
If you measure a proxy, you won’t know when you’ve taken it beyond a point it’s divorced from your true goal and you’re gaming the metric instead of making an impact.
If you measure a single proxy you will miss the bigger picture. If you measure multiple, you will have a hard time deciding what to do when they start to tell you different stories.
All this is not to say that you shouldn’t track any data, I think you should to guide your gut and learn. But it’s futile to strive for absolute objectivity and subjective judgements are extremely useful.
It’s better to be uncertain about being right than certain and wrong.
It’s better to be uncertain about being right than certain and wrong.