Working for a startup with less than 50 people is a life-changing experience. One grows way faster by being exposed to problems that, in big corporations, have departments that take care of them. Aside from that, usually, one works closer to the founders. That gives a good perspective on what it looks like to run a business from the ground up.
Another common trait of working at a startup is that almost every feature and new products come from the founders' vision. It may be a conversation the night before with an early investor or a prospective customer. Or it may just be an idea that will "change the world."
Once the company gets bigger — both in employee headcount and the number of customers —, there's a new VP of Product, Product Manager, or CTO that says:
Folks, this is great. But we need to have a more data-driven approach here. Otherwise, we're just a feature factory, and we aren't validating what we ship.
So, companies hire User Experience researchers. They go all-in in dual-track agile, A/B testing, and more. While all of this makes sense, it also creates perverse incentives. There are good chances that a product hits local maxima due to an obsession with data-driven decision-making.
The three operation modes
Most B2C software companies in winner-takes-all markets operate in three modes:
- Innovation. Here, the focus is to disrupt a market through better, cheaper, or entirely new products. We can think about Dropbox in 2007 or Uber in 2009.
- Fight for #1, #2 or #3. The cat and mouse game of winning a big enough market share to make the business profitable. Usually, there's a lot of "competitor A has features X, Y and Z. We need them now!".
- Optimization. At this point, organizations seek maximum efficiency. It involves reducing operational costs, more automation, or going upmarket and chasing enterprise customers.
Somewhere between the point two and three, companies start to obsess with making data-driven decisions. Everything, from the color of a button to pricing, needs an experiment, user interviews, and so on. There are growth teams at some organizations whose focus is to apply experimentation to maximize efficiency in user acquisition or convert free to paid users.
Gradually, then suddenly
The pressure to improve on daily active users or revenue is enormous. A lot of experiments start, and teams decide accordingly. Maybe they introduce a new step in the user flow to drive more paid subscriptions. Yes, it's inconvenient, but data shows it's worth it. Then, another team in another part of the company is trying a more aggressive strategy to get daily active users. They ended up asking users to sign-up for a newsletter or turn on push notifications. It's a bit annoying, but, again, data says it's the right move.
After rinsing and repeating for a while, all these minor annoyances become a deal-breaker for some users. They start feeling the product they loved is now trying to milk the cow. It's a phenomenon known as gradually, then suddenly. All these experiments, separately, made sense. I mean, the data was there, right? The thing is there's usually no one making sure the overall product isn't degrading. Even when companies track their Net Promoter Score closely, they create a blind spot and open themselves to disruption. And we know that sometimes it takes a whole to come, but when it does, it's damn fast. Some organizations are so into data that they don't care about implementing dark patterns to drive user or revenue growth.
What's the alternative to data-driven decision-making?
Using data and validating ideas through quantitative experiments or qualitative interviews is a healthy practice. It avoids long discussions or opinion-based features, or user-flows. However, we need to be careful when, for instance, we want to optimize user acquisition or growing retention metrics by making it difficult to unsubscribe a service.
One point that usually no one talks about is the fragility of all these optimizations. Sometimes, team A changes the checkout process by tweaking the call-to-action. Then, six months pass, and another team introduces a change that makes the checkout process similar to what it was before. Imagine this multiplied by dozens of teams across a big organization. It becomes a neverending game with a dubious return on investment.
I also believe that too much data-drivenness can hurt innovation in a business. We all remember Henry Ford's quote:
If I had asked people what they wanted, they would have said faster horses.
Let's not give faster horses to our customers. Let's give them a freaking Tesla.
— João