Principles of Estimation
The words “estimate” or “estimation” are often met with skepticism by teams building software. In the software industry, there’s a tendency to get caught up in the “how”, or the right / wrong ways to estimate. We get distracted by the mechanics of it, and don’t always seek to understand the principles behind it.
I have seen teams break down large initiatives into atomic units of work, where each unit attempts to represent a small and fungible (among the team) piece of work - let’s say one day’s worth of work.
I have seen teams consider the practice (of estimation) to be ineffective and abandon any attempts at estimation.
Why bother with estimates? Who needs these estimates and why?
You might hear something along the lines of:
“We estimate so we can generate forecasts! It’s important to be able to communicate the cost involved in building a feature / capability / product. We do this by breaking down the thing we need to deliver into increasingly smaller ‘units’ of work until we’re able to generate high-confidence estimates for how much work each unit is going to require.”
“The more we estimate as a team, the more accurate we will become with our estimates over time. We will understand the team’s throughput, and we’ll be able to offer high predictability over what we can ship, and by when”.
However, this perspective makes some assumptions:
That the effort needed to execute on something is equal between team members. Capacity is often not fungible. Answering the question of “how much effort do we need to build X?” can produce variable answers depending on who is doing the work.
That projects can be broken down with a high degree of granularity, all at once, producing homogenous and atomic units of work. There’s a limit to the granularity with which you can break down a project. Early in the timeline, there can be many unknowns which limits the confidence of the predictions. Trying to break things down into increasingly smaller pieces to produce high-confidence estimates can lead to wasted work if the approach or requirements change. The more upstream the change, the greater the domino effect on estimate changes.
That the point of estimation is only to produce forecasts or plan for capacity, while discounting the value derived from collaborative estimation and conversation, to shed light on mismatched assumptions, misunderstood requirements, missing dependencies etc.
That estimation is a one-time exercise at the start of a project, never to be revised or revisited. That would be like a weather app that made weather predictions a week ahead of time, and then never adjusted those predictions.
That estimates will become increasingly accurate over time. This may be true for teams where the nature of work seldom changes, but in practice I have rarely seen this state (of high precision estimates) being attained and maintained. Teams executing on a lot of novel or innovative work are unlikely to attain this state at all.
All that said, businesses need to make money, and they need to do this by creating value, at a sustainable pace and cost, while trying to do it all more effectively than their competitors. The ability to forecast when that value will be delivered, make prioritization decisions and move nimbly is therefore invaluable.
I don’t have a prescriptive solution (context matters after all), but I try to lean on some principles:
Understand your constraints (time, quality, cost)
Be aware of your assumptions
Know your team’s skills and capabilities
Treat your estimates as forecasts, not timeline commitments. Forecasts evolve, commitments don’t.
Consider associating forecasts with confidence numbers (communicate the degree of uncertainty around your forecasts)
Strive for clarity over the short-term / tolerate some fogginess for the long term
Estimate collaboratively - the value is in the discussion
Lean on a “push” model for communicating forecast updates, particularly forecast changes (think weather “alerts”)
Be curious, not emotional, about forecast changes