Chaos to Waterfall to Agile – The Evolution of Software Development

I spent some time recently in a PMO meeting about project  metrics. Seems that the big bosses want to create some service level agreements based on these metrics. The most interesting part of the meeting was sitting back and watching the process. As an Agile proponent I was amazed at the lunacy of central planning. Each project was on a master project list and resources (humans as I like to call them) were assigned based on their time estimates. These folks were scheduled months in advance. During the meeting, the leader of the PMO said that it was incumbent on the individual project manager to make sure that the assigned resource begins and ends their work as based on this long-term plan. Anyone who has spent any time in software development should realize that the odds of the working as slim to none. Funny thing, every PM in the room, even the head of the PMO, who has a background that includes Agile, thought that this was an appropriate methodology.

The kicker for me was when the head of the PMO stated that a successful project was one were estimated time was plus or minus 20%. A simple calculation told me that I was glad that I was not a PM responsible for a downstream project. If a resource is off by 20% on estimation of a large project, then my odds of getting the time I was promised from this resource is is low. Let’s say that the resource is committed to 160 hours of work on a project upstream of mine. This project can be successful even if this resource goes over 20% or 32 hours. If I only needed him for a week or two,  I certainly will not get the work I need or something else will need to slip. Add scope creep, poor requirements, etc and you have a very small likelihood of success, hence the CHAOS report results for project success.

I was amazed that the PMs would even find this to be a suitable solution until I took a small step back and put the meeting in context. Previous to this heavy-handed approach, the organization’s methodology could be best described as chaos. Thus, anything to bring some order to the chaos was seen as an improvement. In fact, it seems that we as human beings have a tendency to behave like a pendulum with many behaviors. We have a problem so we overreact. We had chaos and “cowboy coding” so now we are going to have a stringently controlled, heavy-handed waterfall approach. In this sense, this organization was following a natural evolution.

The question – is it truly necessary to have the Waterfall approach in between chaos and Agile? Unfortunately I think that the answer is yes. Organizations start out with chaos that can be controlled and effective until they reach as larger size. With the addition of new employees, the old style reverts to uncontrolled chaos and is not sustainable. Next step is to create an organization to “control” the process and the PMO is born. Management loves it because it seems to intuitively provide the proper solution and is better than what we had. After a few years of trying to centrally control production and a few high-profile flops, management realizes what all Agile proponents know, that heavy-handed planning does not work well for most software development projects and they look for the next solution. Enter Agile and Scrum. Scrum is usually chosen because it is easy to comprehend and is better then what we had. And so the pendulum swings again.

Next question – after over 10 years of Agile and Scrum, what will be next on the evolutionary path? Will the pendulum swing back yet again?

Leave a Reply