Recently, I engaged in a discussion with a colleague regarding the Agile vs. Waterfall methodologies and came upon what I like to call The Agile Wall. According to this colleague, there is nothing good about Waterfall. If it isn't Agile, it's crap. And so on. I've encountered this before and it shouldn't surprise me any more but I do think Waterfall has it's place. Now before you go yelling your head off, hear me out...
There is a time and place for everything. In my opinion, it is important not to blindly adhere to a single methodology without taking in a comprehensive view of a new project. Each project is different and there are many factors which should be taken into consideration before defining the process to be applied.
Here is a high level diagram of how complexity of the project should be considered when choosing which methodology will be the correct choice for a project. The problem of this picture is that there are projects that are landing in the "High Complexity" range which do not react well with Agile methodology and end up with poor results. By the same token, a low complexity project which has been bogged down in a Waterfall will probably be just as problematic. The sweet spot for managing a development project is ensuring that complexity is reduced by creating phases and sub-tasks, and applying a hybrid methodology to address development which is beyond the ability for a pure Agile methodology to address.
The real trick is finding the right mix. Agile is a fantastic methodology for very small, low complexity projects, especially when working with small teams. When the project grows to be something more than that, with more stakeholders and more requirements which may be codependent on one another, the advanced legwork of requirements gathering can really end up saving time over the long haul. However, breaking those complexities down into smaller chunks that can be applied in an Agile method can be done to save time and effort overall.
Waterfall can be time consuming and can seem like a waste of time. However, a complex environment can sometimes benefit from a good solid design effort up front. There is also a lot less likelihood of "cowboy coding" resulting in spaghetti code that has been churned out via Agile and Extreme methodologies (have you ever tried debugging some of this stuff??? Ugh!)
So, my advice is that at the beginning of the project, stand back and take a good look at what is to be accomplished. Weigh the triple constraints (scope, time, budget) against the complexity of the project, know your expert teams, and be realistic. Reduce your complexity and apply the methodology to your project, not the other way around. In the long run, you'll be glad you did.
