Since the dawn of computing, humans have sought to estimate how long it takes to build software, and for just as long, they've consistently failed. Estimating even medium-sized projects is devilishly difficult, and estimating large projects is virtually impossible. Yet the industry keeps insisting that the method that hasn't worked for sixty years will definitely work on this next project, if we all just try a little harder. It's the definition of delusional.
The fundamental problem is that as soon as a type of software development becomes so routine that it would be possible to estimate, it turns into a product or a service you can just buy rather than build. Very few people need to build vanilla content management systems or e-commerce stores today, they just use WordPress or Shopify or one of the alternatives. Thus, the bulk of software development is focused on novel work.
But the thing about novel work is that nobody knows exactly what it should look like until they start building. For just as long as software industry has been failing to estimate the work, it's also been deluding itself into thinking that you can specify novel work upfront, and produce something people actually want.
Yet we've also tried that many times before! And nobody cared for the outcome. Because it invariably didn't end up solving the real problems. The ones you could only articulate after building half of a wrong solution, changing direction, and then coming up with something better.
It's time to accept this. Smart programmers have tried for decades, and they have repeatedly failed, just as folks fail today, when we try to cut against the grain of human ingenuity, and insist that software needs estimation.
The solution is not to try harder nor to hope that this time is somehow different. It's to change tactics. Give up on estimates, and embrace the alternative method for making software by using budgets, or appetites, as we call them in our Shape Up methodology.
It turns out that programmers are actually surprisingly good at delivering great software on time, if you leave the scope open to negotiation during development. You're not going to get exactly what you asked for, but you wouldn't want that anyway. Because what you asked for before you began building was based on the absolute worst understanding of the problem.
Great software is the product of trade-offs and concessions made while making progress. That's how you cut with the grain of human nature. It's the core realization that's been driving us for decades at 37signals, and which has resulted in some wonderful products built by small teams punching way above their weight. We've incorporated it into Shape Up, but whether you use a specific methodology or not, giving up on estimates can help you ship better and sooner.