Milestones

There was in fact no correlation between exiting phase gates on time and project success… the data suggested the inverse might be true.
—Dantar Oosterwal, The Lean Machine

 

Base milestones on objective evaluation of working systems.
—SAFe Lean-Agile Principle #5

Milestones Abstract

Milestones mark specific progress points on the development timeline, and they can be invaluable in measuring and monitoring the progress and risk of a program. Traditionally, many project milestones are based on phase-gate activities. In SAFe, however, progress is indicated differently in that it is supported by a fixed cadence of Iterations and Program Increments, allowing teams to acquire a sense of rhythm and help manage outcome variability. Most SAFe milestones, therefore, differ from milestones in traditional project and program management.

PI milestones are based on working systems and thereby support the ability to objectively evaluate the technical or business hypothesis. These occur on the PI cadence.

Not everything, however, occurs on cadence. The complexities of modern software and systems engineering involve multiple factors that rely on external events, third parties, external constraints, etc., that may offer irregular timeline patterns, distinct from the development cadence. These fixed-date milestones are real and material, and must be addressed no matter how Lean and Agile the enterprise is.

In addition, more ad hoc learning milestones help validate business opportunities and business hypotheses, and they help measure emerging business performance of the Capabilities delivered to the market.

When handled properly, each of these types of milestones can bring a required focus to the work, help provide effective governance, and enable better business outcomes.

Details

The development of today’s large systems requires substantial investment—an investment that can reach millions, tens of millions, and even hundreds of millions of dollars. Together, systems builders and Customers have a fiduciary responsibility to ensure that the investment in new Solutions will deliver the necessary economic benefit. Otherwise, there is no reason to make the investment. Clearly, stakeholders must collaborate in such a way as to help ensure the prospective economic benefit throughout the development process and not just rely on wishful thinking that all will be well at the end.

The Problem with Phase-Gate Milestones

To address this challenge, the industry has historically followed phase-gated (waterfall) development processes, whereby progress is measured—and control is exercised—via a series of specific progress Milestones. These milestones are not arbitrary, but they are generally document based, and they follow the apparently logical and sequential process of discovery, requirements, design, implementation, test, and delivery.

But as Oosterwal notes above, they don’t really work. The cause of this problem is the failure to recognize some critical errors with the basic assumption that phase gates reveal real progress and thereby mitigate risk. For example:

  • Using documents as a proxy for solution progress; not only do these create a false sense of security for solution progress, they also drive various measures and metrics, such as work breakdown structures, earned value measures, and others, that may actually impede flow and real value delivery
  • Centralizing requirements and design decisions in siloed functions that may not be integrally involved in building the solution
  • Forcing too-early design decisions and “false-positive feasibility” [1]
  • Assuming that a “point” solution exists, early in the cone of uncertainty, and that it can be built right the first time

The net effect of all the above is that phase-gate milestones have not always helped mitigate risks; instead, a point solution is picked far too early in the cone of uncertainty. Problems follow inevitably, as Figure 1 illustrates.

Figure 1. The problem of phase-gate milestones
Figure 1. The problem of phase-gate milestones

With this backdrop, it becomes clear that a different approach is needed, as is described below.

PI Milestones: Objective Evidence of Working Systems

SAFe provides a number of means to address the problems. In particular, Principle #4 – Build incrementally with fast, integrated learning cycles, especially when used in conjunction with Set-Based Design, provides elements of the solution.

With this approach, the system is built in increments, each of which is an integration and knowledge point that demonstrates evidence of the viability of the current in-process solution. Further, this is done routinely, on the Program Increment (PI) cadence, which provides the discipline needed to ensure periodic availability and evaluation, as well as predetermined time boundaries that can be used to collapse the field of less desirable options. Each PI thereby creates an objective measure of progress, as Figure 2 illustrates.

Figure 2. PI Milestones provide objective evidence
Figure 2. PI milestones provide objective evidence

This is true at both the Program (system integration and validation) and Value Stream (solution integration and validation) levels. Of course, what is actually measured at these critical integration points is subject to the nature and type of the system being built. But the system can be measured, assessed, and evaluated frequently by the relevant stakeholders throughout development. Most important, changes can be made while there is still time to make them, as Figure 3 illustrates.

Figure 3. PI Milestones guide system developers to the optimum solution
Figure 3. PI milestones guide system developers to the optimum solution

This provides the financial, technical, and fitness-for-purpose governance needed to ensure that the continuing investment will produce a commensurate return.

In SAFe, these are the most critical learning milestones that control solution development—so critical that they are simply assumed as credible and objective milestones. In other words, every PI is a learning milestone of a sort. But there are other milestones as well, as described in the sections that follow.

Other Learning Milestones

In addition to the PI, other learning milestones can be used to support the central goal of building a solution that satisfies Customer needs and generates value to the business. It is critical that the value proposition behind a new solution, or a large initiative that advances an existing solution, is treated as a hypothesis that requires conceptualization and validation against actual market conditions. Translating a hypothesis into business demand is the science and art of Lean-Agile product management. It involves a great deal of intermediate organizational learning. Learning milestones can help. For example:

  • Do the new product Capabilities have a market that is ready to pay for them?
  • Do they solve the user problem for the users being targeted?
  • Are the necessary nonfinancial accounting measures available to demonstrate real progress [2]?
  • What revenue can the organization expect?
  • Is there a viable business model to support the new product or capability?

These, and many other business concerns, formulate the basic hypothesis for any large initiative. Learning milestones provide the necessary means to understand the feasibility of the solution and frame the right set of capabilities. Testing a concept of a new capability with a focus group, building and releasing a minimum viable product (MVP), or validating User Experience assumptions for the new functionality are examples of learning milestones. Such milestones do not necessarily occur on PI boundaries and may require significant effort, not only on behalf of the product development organization but also on the part of other business functions in the Enterprise, such as sales, marketing, operations, finance, etc.

Every learning milestone assumes that there is a certain degree of uncertainty that needs to be translated into knowledge and, ultimately, into business benefits for the organization. This requires set-based thinking and the ability to pivot, if necessary, to a different concept of the solution.

Since the outcome of any learning milestone impacts the understanding of intent, milestones are planned incrementally, as Figure 4 suggests.

Figure 4. Learning milestones help evaluate progress toward the goal
Figure 4. Learning milestones help evaluate progress toward the goal

Other elements, including the solution Vision, Solution Intent, and the Economic Framework, also evolve with the learning.

But learning doesn’t stop even when new product capabilities hit the market and start to generate business benefits. Every new capability, and every significant nonfunctional aspect of the system, deserves facts to replace assumptions about anticipated value. In a Lean enterprise environment, learning is an integral part of development, even for mature products. Meaningful learning milestones can help.

Fixed-Date Milestones

Every Lean-Agile enterprise wants to operate with minimum constraints. In part, that’s where agility comes from. The real world, however, has different concerns, and fixed-date milestones are common in both traditional and Lean-Agile development. For example, fixed dates may arise from:

  • Events such as trade shows, customer demos, user group meetings, preplanned product announcements, etc.
  • Release dates that are controlled by other internal or external business concerns
  • Contractually binding dates for delivery of value, intermediate milestones, payment, demonstrations, etc.
  • Scheduling larger-scale integration issues including hardware, software, supplier integration, and anything else where a fixed date provides an appropriate forcing function to bring together assets and validate

There are many more examples. In Lean terms, fixed dates have a nonlinear cost of delay. That is, required system Features become a much higher priority as the date comes closer, as failure to meet the milestones has negative economic consequences. This is directly incorporated into Program and Value Stream Backlog prioritization via WSJF; the “time criticality” parameter gets higher as the fixed date gets closer, thereby increasing WSJF priorities for elements dependent on that date. In any case, any fixed-date milestones should be reflected on the relevant program or value stream Roadmap, so all stakeholders can plan and act accordingly.

Other Milestones

In addition to the above, there are often other concerns required for economic success of product development, such as filing patents, certifying the system, auditing certain regulatory requirements, and so on. In many instances these milestones influence content or priorities of work; they may even alter the development process itself. For example, the need to perform solution certification may increase the transaction cost of accepting a new Release into production and may drive the system builder to seek alternative ways of acquiring feedback before release. Again, any such milestones should appear on the relevant roadmap.

Planning and Executing Against Milestones

An understanding of what types of milestones are required to support value creation may originate from different sources. It may be communicated to portfolios from the enterprise level, or identified during the analysis step in the Portfolio or Value Stream and Program Kanban systems, or even during the planning and roadmapping process for value streams and Agile Release Trains. Eventually, teams will have to create a specific plan of action during the PI planning process, build specific Stories in support of a milestone, reflect the milestone in their roadmaps and PI Objectives, understand and address dependencies with other teams and trains, and negotiate scope and time trade-offs with program stakeholders.

Thereafter, execution of milestones happens incrementally. Progress is demonstrated every PI.

Measuring Success

Successful execution of milestones requires criteria for what “success” means, so there can be value in associating specific measures and Metrics with milestones. For example, a milestone of “capturing X% market share” may require understanding of revenue or usage indicators. A learning milestone of “a search engine being able to reliably identify persons’ names on a web page” could be supported by a limited percentage of false positives across the pages in the “gold collection” of web data. In any case, thoughtful measures make for more meaningful milestones.

 


Learn More

[1] Oosterwal, Dantar P. The Lean Machine: How Harley-Davidson Drove Top-Line Growth and Profitability with Revolutionary Lean Product Development. Amacom, 2010.

[2] Ries, Eric. The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. Crown Business, 2011.

Last update: 29 April, 2016