Lux


Skip Navigation


Lux Headquarters
1008 Western Ave.
Suite 601
Seattle, WA 98104 USA
voice: 206.328.9898
fax: 206.328.9899
info@luxworldwide.com

Project Lifecycles: Waterfall, Rapid Application Development, and All That

View this white paper in Adobe PDF format.

Abstract
The following article reviews several divergent approaches to product processes, and discusses how Lux integrates key elements of these approaches. Lux's project methodology builds on a solid foundation of well-documented project lifecycle management, but incorporates the best of "rapid development" practices.


Project Management 101
In 1975, Fred Brooks looked around at the then-prevailing approach to project management and memorably declared, "Nine women cannot have a baby in a month." [Fred Brooks, The Mythical Man-Month, Second Edition Addison-Wesley, 1995.]

At that time, most of the money invested into software projects was as good as thrown away. Projects routinely went 100% or even 200% over budget, or never delivered at all, or resulted in systems failed to meet any actual need. Projects were thought about almost exclusively in terms of size rather than process. "Project manager" typically meant either the oldest programmer or some guy in a suit who, when it comes down to it, really didn't know much about computers.

Failed software projects are not strictly a thing of the past - the world still contains plenty of inexperienced project managers, incompetent developers, and unrealistic clients - but at Lux, we are proud of our consistent success in accurately estimating the size of Web projects and software development projects and our ability to deliver systems to our clients on time and on budget.

The key to this success is our strong, flexible approach to projects. We have successfully adapted the "waterfall" model to incorporate best practices drawn from various other approaches. This includes elements of approaches such as RAD ("Rapid Application Development") and Extreme Programming that are traditionally considered to be diametrically opposed to the waterfall model. The result is a strong but flexible methodology, incorporating a very large collection of techniques to apply where appropriate.

Every project must thread a unique course between the opposite threats of project chaos and project paralysis. Many of our clients are highly technical companies with their own effective processes in place, and we have a solid history of participating successfully in those processes. The Lux team has collective experience amounting to literally hundreds of projects. We've come to understand very well how to evaluate a project early and to identify what will work for a particular project and client and what won't.

Lux definitely does not subscribe to a "one size fits all" approach to the project lifecycle. Nonetheless, we have some firm views on project process and methodology. We are inherently suspicious of prototyping-centered approaches that claim to work miracles while postponing the difficult decisions until a late point in the project cycle. Prototyping has its place, but when it becomes a substitute for method, it initially gives the illusion of great progress; then one of Murphy's lesser-known laws sets in: "The first ninety percent of the task takes ninety percent of the time; the last ten percent takes the other ninety percent."

In short:

  • Iterative prototyping is simply no substitute for thorough discovery, disciplined development, and well-planned deployment.
  • Prototyping-centered methods produce sites that may look great, but don't quite work right and cannot easily be fixed.
  • A grab-bag of techniques does not add up to a project process.

Lux understands and uses any number of techniques, including iterative prototyping, but we use them within the context of a coherent, disciplined project plan.

The Waterfall, Unmodified
The term "waterfall" dates back to Barry Boehm's work in the 1970s on project management and cost estimation. Probably no one ever used a "pure" waterfall model. Nonetheless, it provides a useful framework to talk about disciplined development of software projects.

In the pure waterfall model, a project is conceived as a series of stages, each of which must be completed and signed off before the next can proceed. The term "waterfall" refers to the idealization that each step is completed, signed off, and then drives the next step. Like a waterfall, there is no going back.

The motivation for this is clear: if you can identify your requirements accurately, you don't spend money specifying and coding programs you don't really need. Similarly, it is a lot cheaper to change a specification than to rewrite code, etc. The theory is that you will solve each problem at the earliest - hence, cheapest - stage where it can be solved. In addition, you will end up with excellent documentation of how your system behaves, how it is structured, and why each key decision was made in the course of the project.

Of course, in practice it is never so simple. Over-adherence to the waterfall model can lead to process paralysis, characterized by:

  • Lengthy meetings where everyone tries to establish unanimity about things they may not yet understand.
  • Inability to proceed until everyone has signed off on a monolithic plan.
  • Rigidity in the face of changing market circumstances.

Thus, the problem for a company like Lux is how to give our clients the benefits of the waterfall model's cost containment, transparency, and control of schedules without its rigidity.

Waterfall

Advantages

Disadvantages

Discovery happens early. Projects follow a predictable cycle. Early identification of what will be difficult or expensive about a project allows go / no-go decisions to be made before too much money is spent.

Great at addressing subtle issues such as scalability, reliability.

Specifications are built into the process.

Client concerns are addressed by requirements gathering and functional specification, so clients don't need to become technical experts.

Developers know that the client's concerns are met by requirements gathering and functional specification, and can feel free to make technical decisions without going back to the broader group.

Relatively high risk of process paralysis.

Poor at handling changing requirements.

Attempting to Thrive on Chaos
Frustration with process paralysis has driven some enterprises to the opposite extreme: to focus on a high-energy development culture, at the expense of process. At worst, this leads to a complete lack of up-front analysis and little or no ability to track the progress of a project. Projects at such companies characteristically degenerate into an unpredictable number of cycles of code-and-fix development, where the end result - if the project ever delivers anything at all - is a system that only a techie could love.

Nonetheless, a number of companies have had some success turning code-and-fix into something workable, usually through some variation on "iterative prototyping." Basically, the idea is that instead of doing a lot of specification and analysis up front, they start from a perhaps somewhat vague idea, do very fast cycles of code-and-fix, and keep handing partially working systems back to the (internal or external) client and users, who identify what is wrong with it, so that each iteration gets closer to what is really needed.

Starting in the 1990s, many such approaches have been proposed, each, it seems, with its own manifesto. Among the better known names associated with this philosophy of development are RAD (Rapid Application Development) and Extreme Programming. The 2001 Manifesto for Agile Software Development is probably the best brief statement of a reasonably mature version of this philosophy. Among other things, the manifesto states:

  • "Business people and developers must work together daily throughout the project."
  • "Working software is the primary measure of progress."
  • "Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage."
  • "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation."

Compared to the waterfall model, this approach (which we will refer to as "ERA", standing for "Extreme / RAD / Agile") has a nearly opposite set of advantages and disadvantages:

Extreme / RAD / Agile

Advantages

Disadvantages

Breaks through process paralysis: when in doubt, prototype.

Barely hampered by changing requirements.

Builds a strong bond between technical and non-technical participants in the project.

Emphasis on short project lifecycles often produces a somewhat useful first-approximation system quickly.

Poor at addressing issues such as scalability, reliability.

Lack of specifications. Because so much of the knowledge about the system is in an oral culture, this can leave a project very vulnerable to departures of individual contributors and can leave a client permanently locked into a vendor, even if the relationship goes sour.

As the Manifesto says, "Business people and developers must work together daily throughout the project." In our experience, most clients who outsource a project do not want intense, daily meetings with their vendor.

Other aspects of the ERA approach are also sometimes claimed as advantages, but we believe these claimed advantages are often illusory:

Claimed Advantages of ERA

But, in Practice This Can Mean...

Allows work to get started while many requirements are still unknown

Allows the illusion of progress when none of the hard issues have been engaged. As we said above, "The first ninety percent of the task takes ten percent of the time; the last ten percent takes the other ninety percent."

Frequent delivery of upgraded software.

A poor approach to mission-critical systems.

A poor approach to systems that a particular population uses over and over. You can't become an expert user of a system that is changing all the time.

Nightmarish support problems.

In all fairness, not every iteration of an iteratively developed system must necessarily be deployed, and there are, indeed, some Web sites where frequent changes and upgrades software are desirable. In that environment, iterative prototyping can be a great methodology, and Lux would not hesitate to use it.

As discussed below, the ERA school has developed many useful techniques that we at Lux employ in our projects. However, we believe that, as an overarching methodology, ERA is a temptation that beckons to people who love to have boisterous meetings and code a lot of throw-away software rather than engage in difficult analysis and build solid, reliable, scalable, well-tested systems.

The Waterfall, Revisited
There are a number of ways Lux modifies the waterfall approach to give it nearly all of the advantages of the ERA approach without the disadvantages. Some of these are almost universally applied among contemporary users of the waterfall approach and are intended to counter the waterfall model's tendency towards rigidity:

  • We allow overlapping project phases.
  • We make calculated decisions to proceed, at times, on slightly uncertain knowledge or informal signoff rather than wait for absolute certainty and formality.
  • We expect that the flow of information from requirements, to functional specifications, to technical specifications, etc., will not be entirely unidirectional, even while striving for that as a goal.
  • We use our good sense and experience to identify when an area of a project is well understood and reasonably independent of any uncertainty affecting the other aspects of the project: allow it to proceed "down the waterfall" ahead of the rest of the project.
  • We expect that there will, indeed be some late discovery and some changing requirements, and we schedule and budget for a reasonable amount of change.

Beyond this, depending on the project, Lux uses any of several variants on the waterfall process. In particular, the "spiral" approach breaks down a large project into a series of sequential mini-projects, with frequent deliverables (documents or software) as in the ERA approach, but without sacrificing rigor and documentation.

Incorporating the Best of ERA into Lux's Process
Besides this appropriate "loosening" of the waterfall model, Lux's actual process incorporates all of the best techniques that characterize the ERA school of thought. For example:

Technique

Applicability

User Interface (UI) prototyping

If a group of stakeholders who will care about the UI are unwilling or unable to attend multiple meetings or closely read specifications, Lux uses a UI prototype to elicit their issues and criticisms. This may even need to be done several times in the course of a project (our version of iterative prototyping).

Joint Application Development (JAD) sessions

The face-to-face meetings so beloved of the ERA folk certainly have their place, especially when there is a need for major knowledge transfer from the client's business experts Lux's team or vice versa.

"Spike solutions" (small programs to test difficult technical issues)

When a particular technical problem can be identified in advance as likely to present difficulties, Lux developers habitually build a small, easily debugged, throwaway prototype program to test proposed solutions to that problem in isolation from other system issues.

"Code Buddies" / "Pair Programming"

There's no question about it, two heads can be better than one. There are times when two developers working together can make faster coding progress - and make far fewer mistakes - than one person behind a closed door.

Object / component organization and multi-level testing

Large systems should not be built as monoliths. Lux makes it a practice to break down large systems into tractable objects and components and to test each object or component thoroughly before attempting to test the integrated system.

Even though we don't wholeheartedly embrace their philosophy, Lux recognizes that there is a lot of wisdom in the Manifesto for Agile Software Development, and we certainly agree with many of their points:

  • "Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done."
  • "Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely."
  • "Continuous attention to technical excellence and good design enhances agility."
  • "Simplicity - the art of maximizing the amount of work not done - is essential."
  • "At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly."

Similarly, there is a lot of wisdom in Don Wells' Extreme Programming (XP), especially in his emphasis on testing, but there is also a lot that strikes us as simply a refusal to do the heavy lifting necessary to produce scalable, and - dare we say it - agile systems ("agile" in the sense of easily adjusting to future change). Except on a very small project:

  • XP substitutes "user stories" for requirements. Even in the hands of an expert, XP's "user stories" don't amount to a thorough use case analysis, and they don't even begin to replace a thorough statement of requirements. The true requirements may involve interoperation with existing systems, security concerns, etc. "User stories" may not reveal these key requirements until the system is already incorrectly built.
  • The XP maxim "Turn a blind eye towards future requirements and extra flexibility" is simply foolish. Yes, it's important to control how much of the budget is oriented toward this future flexibility, but "zero" is seldom the right answer.
  • XP's lack of written specifications will burn you in the end. Maybe not today, maybe not tomorrow, but just wait till you try to train the new hire, or work with a different vendor, or deal with a change in business logic that arises a year after the system is in place.

One successful synthesis of the ERA approach and the more disciplined traditional approach is the Rational Unified Process (RUP). Lux describes our own methodology as a modified waterfall incorporating the best elements of ERA; RUP began as a modified RAD (the "R" in ERA), then tried to incorporate the best elements of the waterfall approach. The results are similar. In particular, RUP incorporates the same strong respect for process, architecture, and documentation as does Lux's methodology. Lux will gladly participate in a RUP-based approach if that is the client's established preference.

Bringing it all together
There's a lot more to building systems than just having a good, flexible lifecycle model. The development of software and Web sites in general, and project management in particular, is an art as well as a science.

  • There is no substitute for experience. An experienced project manager or software developer will have an immediate and correct gut feeling for what will be hard about a particular project and will allocate resources accordingly. Someone without experience will stumble over the issue when the clock is running and the deadline looms.
  • There is no substitute for communication. Lux project managers combine strong communications skills with a solid understanding of the design and development process.
  • There is no substitute for understanding risk management. Correct, early identification of project risks and successful management of these risks is the only way to complete complicated projects on time and on budget.
  • There is no substitute for a broad, mutually respectful, well-rounded team, used to working with one another, continually learning from one another, able to draw on each other's strengths as needed.

Lux is proud to bring together a strong, flexible methodology and a solid team of professionals.

Further Information About Project Lifecycles
Probably the single best discussion of project lifecycles (and many other aspects of software project management) is Rapid Development: Taming Wild Software Schedules by Steve McConnell (Microsoft Press, 1996).

Also very worth reading (and downright entertaining) is The Mythical Man-Month, Second Edition by Fred Brooks (Addison-Wesley, 1995).

Referenced in this article: Manifesto for Agile Development, Don Wells on Extreme Programming (XP), Rational Unified Process (RUP).  


© 2004 The Lux Group, Inc. All rights reserved. This page is provided as a public service to the Web community. As with any copyrighted work, limited quotation with appropriate attribution for purposes of review is permitted. Links to this page are welcome. Explicit permission from Lux is required to otherwise publish, transmit, transfer or sell, reproduce, create derivative works from, or distribute this content, including by incorporating the content into any e-mail. If you wish to reproduce this content, please contact us for permission.

Lux believes that basic information like this should be shared rather than hoarded. Naturally, an article like this only constitutes an introduction to a subject. We hope that if this article has been useful to you, you will consider Lux if you have need for expertise in this area.

Lux
1008 Western Ave. Suite 601
Seattle, WA 98104
phone 206 328 9898 ยท fax 206 328 9899

For permission to reproduce articles:
info@luxworldwide.com

New Business Inquiries:
inquiries@luxworldwide.com

Public Relations:
media@luxworldwide.com

Career Opportunities:
jobs@luxworldwide.com


 
Request a Proposal
Visit our section for new clients. Share information about your project to allow us to produce an accurate quote.
Go Button