2) APM’s Criticism of Waterfall, and 3) Defending the PMBOK® Guide.
Let’s take a step back, and describe how software projects were often done in the past with the waterfall method, and contrast this with the much more iterative approach that’s recommended in Agile Project Management.
In the past, the classic way most of us designed new software applications was to use the “waterfall method.” A precise sequence of phases was undertaken to plan the project, execute, test and close out the project. We first captured our high level requirements – (user requirements, functional requirements or business requirements). In the old way of doing things, this might take a couple of months alone. We then defined the “subsystem specifications,” or the system architecture including file layouts and the modular layout of key program pieces. Again, for a large application, this might take a couple of months. Then the programmers could begin their work, and start the development phase coding all the modules. Depending on the size of the software application this could take a year or even longer. Then the application went through unit testing and integrated testing, and this continued until the application passed all of the acceptance tests. Again, this could take months. Therefore, it might easily be more than a year before the customer actually saw the working application! What has happened in this timeframe, – in this one to two years? Of course, the customer’s requirements have changed. Also, it’s not until the customer has seen the application and used it, that they understand what they truly need. And, even worse, it may not be until the customer actually sees the application that we learn if we really understood their requirements to begin with. So, in a very large percentage of cases, there’s going to be some disappointment when the customer does see the application, and major corrections are going to be needed. Depending on the number of changes, and the level of these changes, it may be a significant amount of time before the customer gets the application they truly need, and significant additional expenditures could be required. We have many of the ingredients of project failure: scope creep, schedule delays, cost overruns and an unhappy customer!
Agile’s Iterative Approach
Again, Agile is meant to address situations where it is a given that at the beginning of the project where the requirements are very undefined: the customer does not really know in any depth what they need, or what they want. Therefore, we will undertake the project in such a way to let these requirements emerge, or be discovered. Therefore, with Agile Project Management (APM), we will use a very iterative approach to tease out these requirements. We must have a strong “product owner” (or product owner team) that can analyze and identify key requirements – or the product backlog – at the outset. We then go through a series of iterations – (or in Scrum – “sprints.”) An initial set of the product backlog is chosen for the first iteration, and this is developed by the project team very quickly – usually within a two week to four week time frame. (Again in Scrum, this would usually be a one-month “sprint.”) In all cases, in APM, it’s a defined “time boxed” short period of time.
The team is given complete freedom and authority on how to best develop (or elaborate) the “stories” (or requirements) for the first iteration The team usually consists of 7 to 8 cross-functional, co-located dedicated individuals, and they (not the project manager) are responsible and accountable for creating the backlog within the defined timeframe. They are empowered, and given complete autonomy and independence on how best to achieve this backlog in the required timeframe. They should not be interfered with during this iteration by upper management, or other stakeholders. The project manager – (in Scrum – the scrum master) is acting principally as a facilitator, and his or her key job is to remove barriers and obstacles that could impede the team from achieving their goals, and ensure that they are not interfered with.
The team meets on a daily basis, – (in a “standup meeting” – or in a “Scrum.”) This is a short 10 to 15 minute meeting where everyone remains standing (resembling a rugby scrum). Assuming the meetings occur at the beginning of each day, each individual describes what they have accomplished the previous day, what they will do today, and what issues they may be facing. (At the standup meeting, the purpose is not to solve any of these issues, instead, they are taken off-line, and the team members independently decide best how to resolve the issues.) At the end of each iteration, the customer – along with the team and product owner – reviews the product backlog (prototype) that has been created, and it is either accepted or not. Ideally, the product backlog created during the iteration is something that could be put into production, and truly used. Also, at the end of each iteration the team goes through a “review or adaptive process” – to catch their breath and get reenergized, and to reflect, learn and adapt. This is usually just a half-day meeting. After each iteration, a new backlog is chosen for the next iteration, and new features and requirements are added on to the previous prototype.
The prototype is progressively elaborated in repeated iterations (or sprints), in a very efficient manner. The customer (product owner) gets to see real tangible products very quickly, and determine the features that are most important and those that are not. In APM, they say we are able to “identify fast failures.”
This Iterative Approach Used in Agile Solves Most of the Problems of Waterall
By using the Agile approach, this will solve most of the problems previously mentioned with the waterfall method:
- We will quickly get feedback from the customer – (not wait the traditional amount of time of more than a year as we did with the waterfall model).
- As Doug DeCarlo says, we will be able to “Identify Fast Failures:” – early in the project we will learn what is not working.
- Determine if we did understand the requirements properly
- Also, oftentimes, it’s not until the customer sees and uses the product that they truly understand what they need.
- Oftentimes, requirements they thought were essential, they realize are not!)
- So, we don’t waste time on non-essential requirements.
- We won’t waste time planning and trying to estimate the unknown.
- We have a cross functional team working together creating the prototype – (or the iteration) – and this is all centered on capabilities, and features that provide functionality for the customer. During the iteration, we come up with a working prototype that is tested and documented. Therefore, the work that would normally occur across several phases in the traditional model – (design – implementation – testing – documentation) – all occurs within a one month iteration, and the cross functional team is responsible for the iteration. Therefore, we do not have the scenario where business analysts define the customer requirements, and then hand these off to the engineers to try to implement who then hand off their products to testing and quality control, and then, finally, we document the product or service. Oftentimes, in this scenario, there will be a number of disconnects and places where the original customer requirements get lost in the translation.
- From Doug DeCarlo’s book, “If a picture is worth a thousand words, a prototype is worth a thousand pictures!”
So, again, most authors of books and articles on Agile Project Management view APM as a complete departure from practices described in the PMBOK® Guide. They describe it as a new revolutionary approach, 180° in the another direction from the PMBOK® Guide, As noted above, in APM, most of the traditional terminology has been replaced by new terms and jargon. This is on purpose! The authors of these new approaches are very aware there are similarities in meaning between the new terms and traditional definitions, but they want to use new terms to emphasize that in APM, teams will be managed in a different way, requirements will be handled differently, and in general, it will not be the old standard way of doing projects. Also, they go to lengths to describe different approaches for managing projects: from planning, through monitoring and controlling, change control and documentation. For these authors, this is a new revolutionary approach.
In APM, it’s a mistake to try to plan the project in detail upfront, and then stick to these plans or baselines. There is less focus on meeting the classic “triple constraints” and holding to these. Instead, according to the Agile manifesto of 2001, the key areas of focus are:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.
In APM, a key theme is that the requirements will be allowed to emerge as we move through the project. Doug DeCarlo describes the traditional method as, “Ready-Aim-Fire,” and the Agile approach as “Ready-Fire-Aim!” We are allowing for continual adjustments and changes to requirements as a move through the project. Ken Schwaber, in his book, Agile Project Management, uses the analogy of planning a car trip to illustrate the APM approach. Perhaps we are planning a trip from Washington, DC to Burlington, Vermont. Do we want to plan the route out in detail before leaving Washington, DC? Probably not with today’s GPS devices, and the ability to monitor traffic in a real-time way. Starting out, we may make some general plans for our route, but allow for adjustments and alterations in real-time during the trip.
Defending the PMBOK® Guide Against This Attack on “Traditional Project Management”
But the PMBOK® Guide also allows that projects could be planned in a very iterative fashion: we often use “rolling wave planning.” This means we will plan in detail things that are occurring right away in the project, and only sketch-in at a high level things that will occur farther out in the future. (We will include “planning packages” in our WBS for these “sketched-in” items.) Later, as we approach the time the planning packages are needed, we will come back – in our second wave of planning or third wave of planning – and fill in the details. So, PMI does not have any problem within an iterative approach to project management. Prototyping is also specifically mentioned as an important “tool and technique” in the process: Collect Requirements.
We can see that much of the Agile practitioner’s criticism of traditional project management is really a criticism of the waterfall method of project management, and is not a criticism of the approach defined in the PMBOK® Guide.
Therefore, one could definitely make the argument that in APM and Scrum, iterations (or Sprints) can be viewed as a special case of Rolling-Wave Planning.
For Both Agile and PMI, Processes From Different Process Groups Are Used in Both Iterations (or Sprints) and Phases.
In Agile, it is understood that within every iteration (or Sprint) a variety of different processes will be used: the team will be allowed a lot of freedom in planning how to do the Sprint, then the team will execute to create the product backlog defined for the Sprint: (the products, deliverables or services), adjustments will be made during the end, and also at the end. This type of activity is very much like doing Monitoring and Controlling processes described in the PMBOK® Guide.
In the PMBOK® Guide the five process groups are also used iteratively in all phases of a project in a very a similar way. (One of the most common mistakes about the PMBOK® Guide is to confuse the five process groups with sequential phases of a project! This is completely incorrect! See page 41 in Version Four of the PMBOK® Guide.) Part of the formal standard is that we should use processes from all five process groups in each phase. Yes, all projects are initiated formally in Develop Charter, but also, all phases must be initiated too. Likewise, just as the project should be formally closed, all phases are also formally closed (as part of Close Project or Phase of Project). We must execute in every phase, since we are creating deliverables in every phase. (In the early planning phases, the deliverables are documents or plans.) And, of course, monitoring and controlling checks on the activities of the other four process groups, so it is always being used: from the very beginning to the very end. Planning is perhaps the most difficult to see being used in all phases. It’s easy to see that it would be used up front in the early planning phases, but why would we still be using planning processes at the end of the project? There are actually two good reasons for this: (1) we are always looking for ways to make improvements, to do continual improvement (Kaizen), and improve our products, plans, baselines or processes; and (2) Rolling-Wave Planning. We may purposefully save some planning for the very end: the “punch list” or “walk through plan.” So, normally, there are even updates to be done to plans and documents at the very end of the project in closing.