High and Low are Equally Wrong
ByIn software development, you’re always being asked to estimate things: How long will the whole project take? Just this feature? What if we changed this feature to remove this aspect? This is all part of the feedback cycle that is fundamental to product creation: We have a certain amount of time and money for a given set of functionality, but if there’s something really juicy and it just takes a little more time, then maybe we adjust, or if we can get a lot of value for a little effort, perhaps we do a little mini release first. The business decisions feed the development estimates which in turn inform new business opportunities.
Getting the feedback wrong can be disastrous: The right functionality late can kill your business; perhaps half a loaf earlier is better. On the other hand, some things the market won’t accept half way so a full loaf it is. The business needs to trust the information it’s getting isn’t random or capricious to make good decisions, and the development team needs to be able to provide a best guess without fear of misunderstanding.
It’s natural to pad estimates with the idea that it’s better to under promise and over deliver – so that means to guess long and come in well early. But in the end, is that really any better?
You’re Guessing, and Possibly Lucky
We’ve been experimenting with the new Evidence Based Scheduling features of FogBugz (which we use internally for managing our software development) and one thing that it highlights quickly is that estimation isn’t good if you’re early, bad if you’re late – it’s about getting your average as close to the mark as you can. Take a look at a graph of most of my estimates:
Ideally, the graph line would have a 1:1 slope, indicating that on average you are accurate. Further, you want your estimates clustered pretty near to the line itself. What you can see from my estimate curve is that I’m uneven – I tend to underestimate shorter tasks (under 1.5 hours) and overestimate longer tasks (and that’s after removing some really bad outliers…). But notably if you draw a ruler on the 1:1 line you’ll see that I’m not even close. Don’t let the hash lines fool you – look at the numbers to see. The thing is, the other developers in our company that are all regarded as skilled, senior developers aren’t particularly more accurate on any one estimate, and the averages work out similarly.
So what?
Why isn’t it a great thing when we beat our estimates? There are several potential pitfalls:
Features based on Effort
We’d all like to believe in the rosy model that our customers ask for features and then we build them into the software, so if we’re done early then it’s a pure win. It’s my experience that it’s much more like a game of Tetris: What features we take on is dependent on how much effort we think they’ll take. Every feature has an amount of effort above which it isn’t worth it any more. When hashing out what makes it and what doesn’t, the effort estimate is a big factor.
If we overestimate the effort of features, then we are slanting the project management decisions away from customer-selected features in favor of the developer’s whims. This is because some features will be estimated to take more effort than they’re worth, and a more invisible internal team dynamic. If a project is doing well on schedule, it’s very human to take advantage of this to try out newer, riskier things, over-engineer a feature, or do other things within the team that would otherwise be successfully argued against because of their effort. In general, the more time available, the more yak shaving the team will do.
Once a schedule is accepted, the business will tend to act on it as fact: Customers that can’t wait for it will end up being turned away and others will be promised a schedule. This is a necessary but painful aspect: Developers are generally optimists and will not want to say no to a customer even though it’s generally not in the best interest of either the company or customer to rush a feature. You want the business to stick with your decisions and not pass the buck on saying no to the customer, but you also need them to trust that it’s a fair trade.
Approach based on Effort
Within the development team, decisions are made at every level on how to implement a feature with an eye towards both the feature’s estimate and the overall project’s status. Even when not explicitly laid out, a team that believes the overall schedule is tight will feel pressured to find ways to reduce the effort on anything they do. This means when deciding between a careful implementation that may take longer but be more scalable or easier to support the team will often opt for a more direct path to completion even if the estimate was based on the more careful approach.
If done as a conscious decision in consultation with the project’s sponsors, this can be a way of bringing the project back on track but really it’s just another way of cutting functionality to make schedule: You’re going to cut out something you intended to deliver (say a more generalized, upgradeable framework for reporting) even though you meet the letter of the requirement in front of you (delivering a few reports). This can lead to nasty surprises for the team down the road when your sponsor’s find out that they didn’t get what they thought they would.
Alternately, if you overestimate one feature it may have put another feature under pressure so a more expedient and risky approach was adopted for it to fit it into the schedule. If the true effort had been known, a different decision could have been made.
Make Your Guesses A Coin Toss
In the end, being early and being late just have different ways they create problems for your development project. Your goal when estimating is to not try to find the estimate that has the highest probability of being sufficient to get the job done but instead the estimate that is equally likely to be high as low. In aggregate if you have enough of these items on your project (say more than 25) then you’re entire project’s estimate should also be just as likely high as low.
There is still a place for the traditional high estimate: When you move outside of the project sponsor and the development team to users that need a guarantee. There the downside of missing a date is much worse than the impact of being early.
On your next project, try out the 50/50 approach and make it clear to both the development team and the business. You’ll probably notice that people develop a more subtle appreciation for the fact that estimates are based on probabilities. This can help you skip over the discussions that aren’t useful about why you are where you are and instead keep focusing on the business goals for your current situation.
Related posts:
