Analyst: 'Water-Scrum-Fall' Is Current Agile Reality

A lot of companies profess to following Agile software development practices, but most of them are still clinging to some of the old waterfall ways of doing things, said Forrester Research Principal Analyst Dave West in an ADTmag.com Webcast earlier this week.

The Webcast was titled "Agile in the Enterprise" (available here on-demand; registration required) and West's keynote presentation focused on "The Promise and Reality of Agile after 10 Years."

That hybrid model of old and new development practices is "the reality of Agile as it's implemented in these organizations that we see at Forrester," West said. "We see that Agile is being adopted heavily by development teams. Teams and individual are picking up Agile and are working on it."

They follow Agile practices such as Scrum meetings, backlogs and rapid feedback on the developing software, he said. "However they're doing it under the constraints of actually, the requirements and the planning were done upfront -- that took three months and was well-documented -- and the fact that we don't release software that often."

But West questioned the effectiveness of that model. "You do get a lot of synergies as a team," he said. "It's a good thing to apply Scrum to development teams. However, remember that a lot of the feedback that you can potentially get on each sprint or each iteration doesn't necessarily help you solve the problem if you can't change those requirements that you originally created in your water-scrum side of the problem."

The result, West said, is that a lot of that time spent on upfront requirements is wasted. Also, he said, the more software teams create, the harder it is to release it, so by having to conform to an infrequent release process that often results from the scrum-fall part of the equation, teams don't present the software to customers as quickly as they would like.

West said planning, preparation and other upfront activities are necessary, but development teams should try to minimize it. "That shouldn't be 50 percent of your project time, or 30 percent of your project time. Try to push that so you do as little of it as possible to allow you to move into a more iterative and incremental lifecycle. And challenge the status quo of releasing software. Challenge that status quo so that you try to release software a little bit more frequently."
Read more

Relative effectiveness

The shift from traditional session-based client/server development to open sessionless and collaborative development like Web 2.0 has increased the need for faster iterations through the phases of the SDLC.[2] This, coupled with the growing use of open source frameworks and products in core commercial development, has, for many developers, rekindled interest in finding a silver bullet RAD methodology.

Although most RAD methodologies foster software re-use, small team structure and distributed system development, most RAD practitioners recognize that, ultimately, no one “rapid” methodology can provide an order of magnitude improvement over any other development methodology.

All types of RAD have the potential for providing a good framework for faster product development with improved software quality, but successful implementation and benefits often hinge on project type, schedule, software release cycle and corporate culture. It may also be of interest that some of the largest software vendors such as Microsoft[3] and IBM[4] do not extensively use RAD in the development of their flagship products and for the most part, they still primarily rely on traditional waterfall methodologies with some degree of spiraling.[5]

This table contains a high-level summary of some of the major types of RAD and their relative strengths and weaknesses.
Agile software development (Agile)

Pros
Minimizes feature creep by developing in short intervals resulting in miniature software projects and releasing the product in mini-increments.
Cons
Short iteration may add too little functionality, leading to significant delays in final iterations. Since Agile emphasizes real-time communication (preferably face-to-face), using it is problematic for large multi-team distributed system development. Agile methods produce very little written documentation and require a significant amount of post-project documentation.

Extreme Programming (XP)

Pros
Lowers the cost of changes through quick spirals of new requirements. Most design activity occurs incrementally and on the fly.

Cons
Programmers must work in pairs, which is difficult for some people. No up-front “detailed design” occurs, which can result in more redesign effort in the long term. The business champion attached to the project full time can potentially become a single point of failure for the project and a major source of stress for a team.

Joint application design (JAD)

Pros
Captures the voice of the customer by involving them in the design and development of the application through a series of collaborative workshops called JAD sessions.

Cons
The client may create an unrealistic product vision and request extensive gold-plating, leading a team to over- or under-develop functionality.

Lean software development (LD)

Pros
Creates minimalist solutions (i.e., needs determine technology) and delivers less functionality earlier; per the policy that 80% today is better than 100% tomorrow.

Cons
Product may lose its competitive edge because of insufficient core functionality and may exhibit poor overall quality.

Rapid application development (RAD)

Pros   
Promotes strong collaborative atmosphere and dynamic gathering of requirements. Business owner actively participates in prototyping, writing test cases and performing unit testing.

Cons
Dependence on strong cohesive teams and individual commitment to the project. Decision making relies on the feature functionality team and a communal decision-making process with lesser degree of centralized PM and engineering authority.

Scrum

Pros
Improved productivity in teams previously paralyzed by heavy “process”, ability to prioritize work, use of backlog for completing items in a series of short iterations or sprints, daily measured progress and communications.

Cons
Reliance on facilitation by a master who may lack the political skills to remove impediments and deliver the sprint goal. Due to relying on self-organizing teams and rejecting traditional centralized "process control", internal power struggles can paralyze a team.
For more