Planning in projects, agile or traditional, is a critical activity, but it is only as good as the information it is based on. The bulk of planning effort should not be reserved for when we know least about a project – at the beginning. Instead planning needs to occur throughout the project and plans need to evolve to reflect the changing realities of projects.

All too often projects begin to diverge from plans and the assumption is that the project needs bringing back on track. Well, perhaps the plan was lousy? The quote “The map is not the territory” from Alfred Korzybski, nicely summarizes the position. When reality diverges from the plan, it is reality we have to rely on. Now, I am not suggesting that we do not track to plans and just let things take as long as they like. Rather, we insert the step of analyzing why the plan and the project have diverged; in software projects the reason is often we had a poor initial plan.

Software is intangible, requirements are difficult to articulate, and rarely is the same system built twice, which makes analogy to existing functionality difficult. These issues can lead to “evaluation difficulties”, where mismatches develop between interpretations of original requirements and the true customer goals.

Secondly, the process of executing a software project is a complex, creative and high risk activity. Unlike many predictable manufacturing projects, writing software in today’s rapidly evolving languages is not a defined, repeatable process. Software development is often an unprecedented, research and development based process. Attempting to create detailed task-oriented plans for software developers is likely to lead to fragile, soon abandoned plans or much project management time spent updating large plans rather than managing the project.

Unfortunately many projects undertake the bulk of their planning too early in the lifecycle, when little concrete data is known about the problem domain, business environment, or team dynamics. The traditional emphasis of planning with project management is illustrated in Figure 1. As indicated by the green shading, planning is predominantly an early project lifecycle activity.

Agile methods accept the project realities of software projects and deliberately make planning a more highly visible, and iterative component of the project lifecycle. In Figure 2 we can see the repeated sets of green shading as planning happens throughout the lifecycle.

Practicalities
Sponsors will understandably want some idea of how much a project will cost and how long it will take to complete and so some complete project estimates and plans will need to be created near the beginning of the project. However, these should be course grained and subject to significant variances – as all they can reliably represent is a mile-wide-inch deep view of the project. Then, only as project details and team performance trends emerge can better defined plans be created.  “Plan the whole project at a high level, and the next iteration in detail” becomes the agile planning mantra.

This concept is depicted in Figure 3.

The green shading darkness represents planning detail. The High level Initial Plan does not have a lot of detail. It outlines the envisioned project timeline and iteration boundaries, but does not contain details about individual features planned to be developed within each iteration. Following planning meetings with the development team and customers, Detailed Iteration Plans are created that list the features (user stories, use cases) planned to be developed for that iteration. A candidate list of features likely for next iteration is held as an Outline Plan, but this list is subject to change as details emerge from the current iteration’s work, new requirements surface, and priorities from the customer evolve.

Figure 4 shows how the plans will look at the end of iteration 2.

Now Iterations 1 and 2 have been completed, a detailed plan has been prepared for iteration 3 and an outline plan created for iteration 4. In addition to completing the iteration plans, the high level plan is updated also, to reflect emerging details, velocities and changes from the project. The map is not the territory, but we should revise the map when we discover new topography on our project travels.

An important difference that may not be immediately clear between agile and traditional planning approaches is Plan Refactoring Effort. As plans increase in size the effort to refactor (update) them to reflect project changes grows at a non-linear rate.

[Figure 5 shows the time in minutes (y axis) to refactor plans consisting of 100, 200, 300, 400 and 500 tasks (x axis). These timings were taken from my own projects and recorded the time required to make a variety of scope, staffing, and business priority changes. Your mileage may vary, but I would be very interested to learn from readers of any more scientific data.]

So, while we intuitively know that it is much easier to update a small plan that update a large plan, as plans exceed 300 tasks the effort to successfully make broad changes increases dramatically. Due to the segmented nature of agile plans (a single high level plan, and separate iteration plans) no single plan ever grows to the size where refactoring is a large effort. Plans can be validated and quickly updated if necessary without breaking them. The same is not true for large plans, which can not be viewed quickly and contain a variety of dependencies that are difficult to track visually as you scroll through a long plan.

Another common problem that leads to brittle, hard to maintain plans is Inappropriate Plan Detail. While planning a project and specifying all the tasks and dependencies there is a tendency to add inappropriate (too much) detail around technical activities. Rarely are project managers the best people to define how technical work should be undertaken. Instead simply listing a placeholder for the completed functionality and letting the domain experts (developers) figure it out leads to more robust plans. Agile iteration plans that simply list features to be developed avoid this inappropriate plan detail risk and the associated brittle, hard to refactor project plans.

Regular updating of project plans is common sense and the PMBOK Guide has two planning guidelines that encourage planning throughout the lifecycle, these are “Rolling Wave Planning” and “Progressive Elaboration”. Rolling Wave Planning states that “planning should be an iterative an ongoing process throughout the project”. Which is an admirable concept, but often very difficult to achieve with large plans. “Progressive Elaboration is a characteristic of projects that are temporary and unique. Progressive Elaboration means developing in steps and continuing in increments.” So clearly the PMBOK Guide recognizes that planning should be iterative and throughout the lifecycle, however because many project managers focus the bulk of their planning effort early in the lifecycle and create large plans that are difficult to refactor, these recommendations are rarely implemented.

Summary
Instead of attempting to create huge, detailed plans that quickly diverge from reality and are difficult to maintain. We are be better served by leveraging developers individual ability to manage local complexity and create lightweight plans that list features, but do not attempt to dictate implementation steps. In doing so we create plans that can embrace the inevitable changes of software projects and regularly produce small plans for the upcoming features we do know about. The interesting thing is that our agile plans support both the Progressive Elaboration and Rolling Wave concepts of the PMBOK better than most traditional project plans do, yet there is little acknowledgement of this alignment from either party.