On the picturesque island of Oahu, Hawaii, is an incredible waterfall. The Waipuhia Falls is like most waterfalls in that the water starts at the top, reaches a cliff and begins to fall. However, as the environment around it changes, the water direction changes. Strong north-easterly winds cause the water to reverse and move upwards, giving it the nickname the 'upside-down waterfall'. The water often falls back on itself at the source, starting the process again.
Many people refer to organizations as either doing waterfall or something under the Agile umbrella, for example, Scrum. Many people refer to waterfall as if it is the antithesis of Agile and Scrum.
"Be careful of calling Product Backlog Items requirements, you'll fall back into waterfall."
"Don't do too much planning in Sprint Planning, we don't want this to become a waterfall project."
"We need to be able to adapt to change, otherwise we're just a waterfall team."
Upon further inspection of Agile, Scrum and waterfall we see that this is a false dichotomy. Instead, we can assert that waterfall is a loosely defined approach to development that, to some extent, will always exist.
A Closer Look at Waterfall
The exact evolution of the waterfall approach is blurry. Winston Royce talked about a set of sequential steps to development in his 1970 paper, Managing the Development of Large Software Systems. Royce describes a set of sequences: System Requirements, Software Requirements, Analysis, Program Design, Coding, Testing, Operations, in which each phase trickles down in to the next. The implication is once System Requirements are gathered, Software Requirements can be collected, and once that's done, Analysis can begin, and so on.
Interestingly, Royce wrote, "the implementation described above is risky and invites failure." He encourages an approach where the phases form an "iterative interaction" between each other. He even posits that the most optimal and least risky approach would be to build an "early simulation of the final product," built upon the same sequences of development. One could describe this as a series of mini waterfalls, where, depending on the changing environment, some may become "upside-down" and feed back into previous waterfalls.
In all of this, however, Royce never used the term waterfall.
Bell and Thayer's paper published in 1976, Software Requirements: Are They Really a Problem?, appears to have made the first connection between the sequential steps laid out by Royce and the term waterfall.
Bell and Thayer introduce waterfall in reference to 'Figure 1' in Royce's paper, which describes the "two essential steps common to all computer program developments, regardless of size or complexity". Those steps are an analysis step and a coding step. What stands out here is their existence is regardless of size or complexity. That is, in some sense this concept of a waterfall flow of work must always occur. This makes sense. When one's shoe laces are untied, one recognizes the problem, decides what to do, bends down and ties their shoe laces. This happens in a fraction of a second, but the general flow is: Analyze the situation and then do the work.
Naturally, organizations try to bring uncertainty and complexity under control with extensive processes and policies so that the desired results are obtained. These processes and policies wrap around the waterfall flow of work, which cause delay, and ironically, often impede optimal results. For example, a slow approval process causes delay in moving from the design to development sequences.
In a process light, self-managing environment, which you would hope to find in a Scrum Team, for example, this transition from the design to development sequences might be seamless - to the point it may appear invisible. It is important to recognize, for the process heavy organization, waterfall is not the cause of delay, the process and policies in places are. This is why, when a team or organization adopts Scrum, very little change is realized because the same processes and policies are likely to still be in place.
A Closer Look at Agile
While the spirit of continuous improvement, organizational growth and business agility has existed for much longer than The Agile Manifesto, the term ‘Agile’ was only formally acknowledged and used for this purpose since the manifesto’s creation in 2001. Since then, the concept of waterfall took on a bad name. It is not uncommon for people to refer to waterfall as the reason why organizations and their products fail. Waterfall is often used as a pseudonym for organizations that do not fit some arbitrary threshold of Agile maturity.
The Agile Manifesto, however, is simply a mission - to uncover better ways of developing software - supplemented with a set of values and principles. The manifesto defines no roles, no sequences of events, and no rules about the flow of work. It makes little sense to consider the relationship between waterfall and Agile as either/or.
The Department of Defense referred to building products as either "using Waterfall" or "using Agile" in Design and Acquisition of Software for Defense Systems, published in February, 2018. In it, they describe waterfall as something where work moves "in largely one direction". Note "largely in one direction", but not limited to. However, it describes Agile as "continuous iterative development, where a team develops software in smaller blocks that can be incrementally evaluated by a user community." Does work not flow in some sort of waterfall-like pattern in these smaller, incremental blocks?
Throughout the 1980s and the 1990s, the term waterfall was used here and there, but again, its exact usage is hard to track. The Project Management Institute (PMI), who first released the Project Management Body of Knowledge in 1996, did not start using the term waterfall until 2013 (in its 5th edition), 12 years after the Agile Manifesto was written. The latest PMBoK now distinctly separates waterfall and Agile. It seems that waterfall as a term became more popular in recent years simply as something to contrast with Agile. It would not be unfair to say Agile advocates tend to put practices they like in the "Agile" bucket and everything else in the "waterfall" bucket.
A Closer Look at Scrum
"Scrum is a lightweight framework that helps people, teams and organizations generate value through adaptive solutions for complex problems." - The Scrum Guide
Scrum originated in the 1990s and has seen several adaptations - mostly to simplify it - since then. It is held together by its three accountabilities, three artifacts, and five events, all providing transparency to help with the inspection and adaptation of product development.
The essence of Scrum is found in its rhythm, most notably with the Sprint event. "Sprints are the heartbeat of Scrum, where ideas are turned into value," and despite the Agile community's tendency to cower at the word 'project', the Scrum Guide also describes a Sprint as something that "may be considered a short project."
A Sprint can be no longer than one month. This is relatively short compared to projects one might be more familiar with at larger organizations. The benefits to a shorter project include the reduced risk accrued by bringing the project to a close sooner, providing formal opportunity to inspect the results and adapt what should happen next. When projects are longer, there is a higher risk of change in the market that may negatively impact the value of the purpose or goal of the project and therefore, the product.
A Scrum Team has only two commitments active at any given time: The Product Goal, or the long term objective that gives each Sprint direction and focus; and, The Sprint Goal, the single commitment for the Sprint, which gives the Scrum Team direction and focus for that short project.
The PMBoK defines a project as "a temporary endeavor undertaken to create a unique product, service, or result."
A Scrum's Sprint is a fixed length event, rendering it a temporary endeavor. The unique result, and the single requirement for a Sprint is the Sprint Goal. While the items that get done to achieve it may flex, the Sprint Goal is the sole requirement for that Sprint.
Waterfall implies that requirements are defined in its early sequences. With Scrum, the Sprint Goal is defined in the earliest event, Sprint Planning. In fact, "The Sprint Goal must be finalized prior to the end of Sprint Planning."
Development pursues and the project is brought to a close at the Sprint Review and Sprint Retrospective, where the results of the Sprint are discussed and decisions of what to do in subsequent projects are considered.
Through these definitions and interpretations, a Sprint can indeed be seen as a project in which the waterfall flow of work can be and is experienced.
Conclusion
To simplify:
1. Agile (the Agile Manifesto) doesn't define an order or sequence of events. To compare it to waterfall is comparing apples to oranges.
2. Scrum only minimally describes the order of events within a project (a Sprint). What is described fits with the mapping one might expect of a waterfall flow of work.
3. The sequences (Requirements Gathering, Design, Development, etc.) in product development can be, and always have been able to be, revisited when change requires it.
In practice, the organizations with heavy process and policy often discourage it, make it slow, wasteful and costly, but change, nevertheless, has always been a possibility.
4. Each increment of work, no matter the size, can be seen to take on a waterfall flow of work, even if the flow of work is very small or sometimes restarted to adapt to change.
To describe organizations as either employing waterfall or Agile/Scrum is inaccurate. To present them as a dichotomy is misleading. I encourage people to stop referring to organizations as 'waterfall organizations'. This categorization is not only technically redundant, but more importantly, possibly detrimental as it may cause people to focus on the wrong issues.
Instead, we should be recognizing that all organizations have processes and policies that impede the natural flow of work. We should focus on removing wasteful processes and policies wrapped around methodologies and frameworks, rather than simply choosing a new one and becoming frustrated when very little benefit is realized.
If your organization refers to its product development projects as 'using waterfall', that doesn't necessarily mean you're doing things wrong. Look at the policies and processes that impede the flow of work, resulting in a longer time to learn and a longer time to return value. Look at the size of the waterfalls and consider if shortening them would help mitigate risk.
Comentarios