There was in fact no correlation between exiting phase gates on time and project success … the data suggested the inverse might be true.

—Dantar P. Oosterwal, The Lean Machine

Principle #5 – Base milestones on objective evaluation of working systems

The Problem with Phase-Gate Milestones

Developing today’s large systems requires substantial resources—investments that can total millions and even hundreds of millions of dollars. Developers and customers have a mutual fiduciary responsibility to ensure that new solutions investments deliver the necessary economic benefits. Otherwise, what is the reason to proceed?

Stakeholders must collaborate in ways that help ensure the potential to realize the prospective economic benefit throughout the development process versus engaging in wishful thinking until the end. The industry has generally applied a sequential phase-gated (waterfall) development process to address this challenge, measuring and controlling progress through a series of specific milestones.

These phase-gate milestones are not arbitrary. They follow a seemingly logical and sequential process: discovery, requirements, design, development, test, and delivery. Of course, it hasn’t always worked out all that well, as Figure 1 shows.

Figure 1. The problem with phase-gate Milestones
Figure 1. The problem with phase-gate Milestones

The cause of the problem is the failure to recognize the four critical errors in the assumption that phase gates reveal real progress and thus mitigate risk:

  • Centralizing requirements and design decisions in siloed functions that do not actually build the system.
  • Forcing too-early design decisions and, thereby, false-positive feasibility. [1]
  • Assuming a point solution exists and the system can be built correctly the first time. This assumption ignores the variability inherent in the process and provides no legitimate outlet for it. Variability will find a way to express itself.
  • Making up-front decisions creates large batches of requirements, code, tests, and long queues. These long queues lead to large-batch handoffs and delayed feedback. (Principle #6 – Make value flow without interruptions)

Base Milestones on Objective Evidence

Clearly, the phase-gated model does not mitigate risk as intended. A different approach is needed. This principle — build incrementally with fast, integrated learning cycles — provides a strategy to address this dilemma.

In this approach, the system is built in increments, each of which is an integration point. Each integration point demonstrates evidence of the feasibility of the solution under development. Unlike phase-gated development, every milestone involves a portion of each step—requirements, design, development, testing—producing a full increment of value (Figure 2).

Figure 2. Milestones based on an objective evaluation of working systems
Figure 2. Milestones based on an objective evaluation of working systems

Further, this is done routinely on a cadence (Principle #7 – Apply cadence, synchronize with cross-domain planning), which provides the discipline needed to ensure periodic evaluation at predetermined time boundaries. This regular evaluation quickly collapses the field of less desirable options.

The nature and the type of system under construction will determine what is measured at these critical integration points. But the system can be measured, assessed, and evaluated by the relevant stakeholders frequently throughout the solution development life cycle. Doing so provides the financial, technical, and fitness-for-purpose governance needed to ensure that the continuing investment will produce a commensurate return.

Based on this principle, SAFe makes some milestones explicit. For example, the PI system demo is designed specially to deliver objective metrics on progress, product, and process, as Figure 3 illustrates.

 

Figure 3. An objective look at progress, product, and process metrics at every PI system demo
Figure 3. An objective look at progress, product, and process metrics at every PI system demo

Progress metrics determine if the organization is on track to achieve the current mission for the solution. Product metrics provide objective data on how the solution is performing in the market and insights into future needs. Process metrics bring visibility to improvement items that are currently in flight. Together these objective metrics enable stakeholders to frequently assess and evaluate business and technical progress and decide whether to continue to invest in the solution or allocate investments elsewhere.

 

Learn More

[1] Oosterwal, Dantar P. The Lean Machine: How Harley-Davidson Drove Top-Line Growth and Profitability with Revolutionary Lean Product Development. Amacom, 2010.

 

Last update: 14 March 2023