Estimating.
As developers (or however we want to call ourselves), one of the hardest and thankless tasks we will face is a surprising one: Estimations.
In this article, I will explore and answer the questions:
- Why is it so painful, and why are we always wrong?
- Why does the business need our estimates?
- How can we be better with estimations?
Why so much pain?
Let’s face it, estimating is painful because we are awful at it.
We tend to be overly optimistic and too focused on short term / immediate problems (at least that is what the pop science I keep reading tells me)
Because of that, we will tend to estimate way too short even when we try to be pessimistic. Based on personal observation, there is a caveat here; we tend to be too pessimistic with tasks that take less than a couple of hours.
On a typical estimation task, we will:
- try to outline (in writing or in our head) a rough outline of the tasks/problems we need to solve,
- Imagine roughly how much time each of those will take us, and
- Add some time for any ‘unknown’ issues that might arise.
But, as discussed before, we will tend to underestimate the time it will take for our tasks and the unknown that we will face.
A current proposes that if we break up the tasks to be small enough, we will manage to be more accurate. The problems I see with it is that:
1- At the beginning of a project (when we are asked to estimate), we usually do not have enough knowledge to identify all the tasks
2- We will compound and break up our flawed estimations into smaller chunks.
Why does the business keep pestering us for estimations?
To understand and answer this question, let’s try to get into the head of a typical ‘business person’ (product, operations, or whoever takes into their hands the roadmap).
As much as we usually deal in uncertainties, they need to have some certainty. They need to know, among many other things, the answer to:
how many resources will be tied to this project, and for how long?
They know that we cannot be accurate, they accept that, but they need to plan. And for that, the business needs at least a ballpark assessment of how long a given task will take.
Don’t get me wrong, if you consistently and significantly underestimate, they will not appreciate it and call you on it. And if you do the opposite (overestimate), the business will push back and make you reduce your estimated times.
how can we be better?
There are several schools of thought to answer this question from my exploration. Let me draw first a non-exhaustive list of them and then tell you what my recommendations are:
Break everything into tasks that are no longer than a day (SCRUM)
This approach works under the assumption that we can produce meaningful changes in a short period (a week or two) and that we can change priorities after each of those periods.
Each week the team will decide what small tasks to work on, knowing that no task should take more than a couple of estimated days.
The more significant issue I see for this approach is that it breaks when estimating long-winded projects (think of a change of branding or a complete redesign/overhaul of a website). On top of that, if you run into an unknown problem on one of the ‘small tasks’, it will ripple into the next ones.
Don’t estimate (Shape Up)
This you might not be familiar with. The broad lines are that we do not estimate. We choose how much time we are willing to spend on an issue instead. If it remains unsolved after the allotted time, we scrap the project and re-define it. (do go read shape up for a better understanding)
The main problem I see here is that it puts a lot of pressure and uncertainty on ‘the business. First, it is tough for them to say this is only worth X amount of time. But more importantly, it adds the scary proposition: we might throw away the work if we don’t have enough time. And yes, it is a sunk cost, but still, it is scary.
Just add X%
I believe this to be kind of a no-no. It does not really solve the problem; you are just trying to account for uncertainty and unknowns, but you don’t account for them realistically by adding a fixed amount.
Recomendations (let’s estimate better)
So, as I hope I established, the need for somewhat precise estimations is not going away. How can you, as an engineer, get better at it?
Change the scale
The scale you use implies the precision you are giving. Let’s say that a project took you 17 days to complete, but you estimated it to take fifteen days. It would be more aggravating if your estimation was expressed in days instead of weeks.
A good rule of thumb is that if something takes you more than 2 units of time, you pass it to the next unit on the following scale:
- hours
- days
- half-months
- months
- half-years
- years (although if you are estimating in years, you have more significant worries)
Reflect on your estimations
When you finish a project, look at how long you estimated it to be. Look at how wrong you were and why. Try to reflect on what when wrong when doing your estimations.
This seems trivial, but it takes some effort to do that reflection. It will allow you to draw lessons that you can apply to your other estimations.
Be upfront with uncertainties
If you don’t know how long something will take, but you are still pressed into an answer, try to give a couple scenarios for best, average and worst cases:
If X happens, we will probably finish in a couple of days; If Y goes wrong, we will take a week instead; If both X and Y go wrong, this could take up to a month. (Do give an approximate proportionality to each scenario.)
Conclusion
Estimations are not going away; business DOES need them, so we need to better speak their language and estimate within reasonable bounds.
Our responsibility as engineers is to answer business when working on a product or feature.
