Better planning and estimating

There are countless cases where major delays in product development are caused by poor estimation-related decisions. When it comes to both, high-level (long term features) and low-level (user stories, daily tasks) planning, developers tend to underestimate the work.

There are few main reason for that:

•        not taking into consideration all factors

•        lack of knowledge

•        being over-optimistic

•        not identifying all risks

There are some techniques that can make estimates more accurate.

Big picture

Before any estimation make sure you take into consideration all factors and see the big picture. If you are a developer don’t think only about coding part, consider all steps of development: analysis, coding, code review, manual testing, automated tests, build, deployment.

Example questions you should ask yourself: Are the business requirements clear and well-prepared? Are there any pitfalls in coding? Is it easy to test? What can go wrong in testing? Are there any automated test cases that can fail? What are the chances build will fail?

When you consider all that factors you will have better understanding of whole process.

I know that I know nothing

 Donald Rumsfeld, United States Secretary of Defense, stated:

 „There are known knowns; there are things we know that we know.

There are known unknowns. That is to say, there are things that we now know we don’t know.

But there are also unknown unknowns. There are things we do not know we don’t know.”

Obvious is that the more you know about a feature you’re about to estimate the better. But rarely you will know as much as you would like to beforehand. When you recognize things you don’t know it’s also a big plus, sometimes it’s enough to know where to seek that knowledge e.g. whom to ask about that project module or where to learn about new technology. The most dangerous are the unknown unknows – the mysteries that will reveal itself while you’re progressing your task and that will slow you down. Sometimes task seems easier that it really is because one doesn’t know how little he or she knows.

To prevent such scenario necessary is to take some time and learn as much as needed about estimated topic before estimation. The more unknowns are revealed the better. Take a look into code, try to figure out what technically needs to be done, make sure current architecture will make it easy to code. Talk to more experienced in that field devs to ensure you haven’t missed any crucial aspects. Keep in mind performance and security. Identify what skills and technologies you will need to learn and how long will it take.

How much time to spend doing such investigation is highly individual and depends on both your current knowledge and size of the new feature. It can vary from 15 minutes when you just make sure you know how it should be done, to 2-3 days when you need to provide high-level estimates on feature in field that you are not really familiar. Time spent on pre-estimation preparation is always time well-spent.

When after preparation you still have a feeling that there are unknown unknowns the estimate should be higher.

Take a look from the outside

 Being over-optimistic is a widely common issue not only in software development estimation but in every field that requires planning. Many psychological studies have shown that people tend to put their estimates closer to maximally optimistic scenario rather that realistic, especially when it comes to time required to fulfill some task.

To prevent planning fallacy there comes statistics.

Sometimes the best solution for estimation is to take step back and see how similar tasks were planned and executed in the past. Whatever your work-tracking and planning tool is try to find comparable tasks and try to learn from them. How many story points your team have planned for each iteration and how many have delivered? What are postpone rates? What kind of tasks are mostly under-estimated? What about other teams? The more data you collect the better your future estimates will be. Maybe you will find out that 25% of tasks were under-estimated by 50% – that will be a good reason to estimate higher next time. Try to categorize tasks, it will help to find correlations. Finding out that most of GUI-related work was delayed may bring profit in next planning sessions. 

Risk management

A good practice before task estimation is to recognize and categorize all possible risks.
To identify risks we need to think what can go wrong on every possible level (technical, project, process, organizational)

Each recognized risk should be categorized by probability of occurrence and severity (impact). For that we can use keywords like low, medium, high or use 0-100 percentage values if more detailed analysis is needed. To visualize it, risks can be marked on chart shown below:

Risk management chart with 3 risks marked

Risks identified in green area are less serious that the ones closer to red sector. The more red-area risks the higher the estimate should be considered.

Risk may be any factor that brings uncertainty that feature will not be fulfilled in a given amount of time. Risks can occur on each step on the process and are related to every aspect of future work e.g.: lack of (any kind of) resources, business requirements being subject to change, third party software incompatibility.

Planning and estimating are commonly underestimated skills that require some experience like any other kind of tasks. Fortunately there are practical techniques that make it easier to master.