A Proposed Approach to Estimating

“Estimation (or estimating) is the process of finding an estimate, or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable.” (Wikipedia)

Estimating how long a task or project will take is a critical part of the software development process, and unfortunately it is extremely difficult. Why? Because it is essentially a guessing game, based on assumptions and information we don’t yet have. Yet the consequences of poor estimation can be quite significant. On a personal level, it can lead to embarrassment, stress and a damaged reputation, as you fail to meet deadlines which were essentially set by yourself. At the team or organization level, poor estimation may ultimately be the difference between success and failure in a project. If you under-estimate a task, you will have to sacrifice quality or cost to meet your deadline, or alternatively you will deliver late – bad news either way.

Your client or business obviously wants software to be delivered as quickly as possible, without compromising on quality. As professionals therefore, we should endeavour to provide estimates which are as low as possible, without being unrealistic. In other words, if we honestly believe that a project will take six months, we shouldn’t say it will take a year “just to be on the safe side”. I have seen some developers recommend the practice of taking a guess, and then tripling that estimate. A better approach I believe is to take your time when producing an estimate, and do your best to predict the task in detail, however unpredictable it may seem. The fact that estimation is difficult should not be an excuse for not trying to do it well.

I will outline an approach to estimation here which should at least provide food for thought. It is based primarily on one principle: when it comes to producing accurate estimates, we need to take our time and be thorough. In practice, this principle is rarely followed.

The approach assumes that you have been asked to produce an estimate only – not a specification, a design, or any other kind of official document. Typically, the request will have come from someone who is trying to weigh up the value of pushing ahead with a task or project. I have been in this position many times, and have often made the mistake of spending approximately 10 seconds thinking before coming back with an answer. What I am trying to do here is propose a better and more systematic approach.

A second assumption I am making is that you are being asked about a task or project which will take a day or more. Anything smaller than that and it is probably not worth trying to be too clever with your approach.

The process consists of 6 steps.

Step 1 – Isolate Yourself

When you are put on the spot, it is difficult to think clearly. It is also difficult to think about a complex problem in your head, without drawing, brainstorming, scribbling and making lists. Furthermore, a reliable ‘gut feeling’ tends to come only from adequate experience, which is rare given the inherent uniqueness of each software project.

Therefore, your first task when asked to produce an estimate is to isolate yourself from the person making the request. This doesn’t mean you have to lock yourself away in a room by yourself. It just means that you need to tell that person that you will get back to them. Don’t give them an answer immediately just because that is what they expect. You won’t actually know how long you will need to produce your estimate until you have completed steps 1 to 4, but this can be done in a few minutes, so at worst, you will be able to get back to them very soon and tell them how long you need to produce an estimate. There may of course be a problem here that comes from pressure to provide an answer quickly (“just give me a ballpark figure”). If they won’t take no for an answer, then go ahead and triple your gut feeling, but explain that if you are allowed some time to think about this more carefully, you will be able to provide a better estimate.

Step 2 – Choose an Order of Magnitude

Based on your gut feeling, pick one of the following four categories for how long you think the task will take:

• 1 to 5 days
• 1 to 4 weeks
• 1 to 6 months
• 6+ months

Step 3 – Choose an Importance Factor

Make a call on how important you feel it is that your estimate is accurate. In some cases, there is not much riding on the accuracy of your answer. You may know the person well, and know that they will not hold you to your answer and that they understand you are just giving them a rough idea. On the other hand, you might suspect that you will be held to whatever answer you give, and that it is very important that you provide the best possible estimate.

• 1 = Low importance
• 2 = Medium importance
• 4 = High importance

Notice that the ‘High importance’ factor is assigned a value of 4, rather than 3. This is to reflect the fact that an estimate deemed as highly important is probably twice as important as one that is regarded as of medium importance.

Step 4 – Calculate How Long to Spend

Based on your ‘order of magnitude’ estimate, assign a base time for how long you need to spend on your estimate, as follows:

• 1 to 5 days = 30 minutes
• 1 to 4 weeks = 1 hour
• 1 to 6 months = 2 hours
• 6+ months = 1 day

Now take that base time and multiply it by the importance factor value from step 3, and you have a value for how long you should spend on the estimate. For example, if your feel a task will take between 1 and 4 weeks, and you assign the estimation task a medium importance factor, then you should spend 2 hours on your estimate.

How long you spend on an estimate should be a reflection of how complex the task or project is, combined with how important it is that your estimate is accurate. The longer you spend on an estimate, the more likely it is to be accurate. In this respect, I don’t believe an estimate is ever really ‘done’, and a sensible approach is to decide up front how much of your time and energy you are going to devote to trying to get your estimate right.

Step 5 – Estimate

Now you know how long you want to spend on your estimate. I would recommend breaking up this time into the following steps, to ensure you make the most of this time. These steps are based loosely on the vertical planning process described by David Allen in his book Getting Things Done.

1. Describe the Project or Task in a Sentence

This will help you to take a step back and briefly consider the big picture, and what it is you are trying to achieve.

2. Describe the End Result

How will you know when the project is done? Write up your answer to this question. Depending on how much time you have available and your preferred approach, this may consist of a short paragraph of text, or a long list of informal acceptance tests. By thinking ahead to completion, you are more likely to identify challenges which you will encounter along the way.

3. Brainstorm

Using a list or mind map, get down on paper (or on your machine) everything you can think of relating to the project. What could go wrong? What tasks are involved? What is relevant? What are your general thoughts?

4. Technical Design

This is the most difficult part and again, the level of detail you go into will depend on how much time you want to spend. You may want to divide your task into horizontal tiers, thinking in terms of the database, data access layers, middleware and user interface. In each of these layers, what will your solution look like? If you only have a few minutes, you may just sketch a couple of diagrams or make some lists. If you have longer, you might produce some more comprehensive UML diagrams, or do whatever else you would typically do to design a system.

5. Identify Sub-Tasks and Estimate

As you produce your technical design, you should naturally begin to identify sub-tasks, which you may break down further still. As you identify these lower level tasks, you can assign estimates to each of these, and add up all your low level estimates to produce your overall estimate. In terms of estimating these low level tasks, you may choose to use a technique such as PERT estimating, or you may even get other people involved and play planning poker. Even if you decide to use your gut feeling, as I generally do at this stage, you are more likely to produce a better overall answer than if you try to use gut feeling at a higher level.

Step 6 – Negotiate

You have been through the process and you have an estimate. If you are lucky, when you provide this estimate to the person who requested it, they will accept it and your estimation task is over. If you are less fortunate, they will pull a face and question your estimate. Obviously, they were hoping the project would not take so long. If this happens, then you should have some evidence to hand to justify your estimate. Show them your technical design, your sub-tasks and their corresponding estimates. If they are still unsatisfied with the estimate, then it may be appropriate to negotiate based on the Project Management Triangle. If your estimate is still not accepted, then at least you know that you have done your best to provide an honest and accurate answer.

Final Thoughts

Estimation is hard, and as I always maintain in every aspect of software development, there is no silver bullet. There is no single approach or technique that is going to work for everyone. What I have done here is to try to come up with an approach which I believe will help me become a better estimator, and therefore a better professional. Obviously, software projects overrun due to unanticipated obstacles. We need to take our time to try and anticipate the obstacles which we will inevitably face, and not rush the estimation process. When you are asked for an estimate, replying immediately with a number is generally a bad idea. Our brains aren’t powerful enough to process all the relevant information that quickly. So accept the fallibility of your brain, and take your time when estimating.

The post A Proposed Approach to Estimating appeared first on The Proactive Programmer.