Assume variability; preserve options.

—SAFe Principle #3

 

Set-based Design

Set-based Design (SBD) is a practice that keeps requirements and design options flexible for as long as possible during the development process. Instead of choosing a single “point” solution upfront, SBD identifies and simultaneously explores multiple options, eliminating poorer choices over time. It enhances flexibility in the design process by committing to technical solutions only after validating assumptions, which produces better economic results.

System development can be described as a process of continuously converting uncertainty to knowledge. No matter how well a system is initially defined and designed, real customer needs and technological choices are both uncertain and evolving. Therefore, understanding how a system needs to be implemented must adapt over time.

Details

SBD maintains multiple requirements and design options for a longer period in the development cycle. As the timeline advances, SBD uses empirical data to narrow the focus on the final design option. Through this approach, system builders can assume variability and preserve options for as long as possible, providing maximum flexibility.

Conversely, Point-based design commits to a set of requirements and a single design strategy early in the process. It often leads to late discoveries that require substantial rework as the deadline approaches. This may necessitate shortcuts, quality compromises, and—worse—missed program commitments and deadlines. Using the Continuous Delivery Pipeline cadence to provide feedback and learning, Set-based design maintains multiple design options over a longer period in the development cycle.

SBD is an important practice for economic efficiency in Lean product development, described further in references [1] and [2]. Figure 1 shows the conceptual difference between set- and point-based design approaches.

Figure 1. Point-based design commits to a design up front. Set-based design maintains multiple alternatives for a longer period.

Increase Economic Efficiency with Set-based Design

Employing SBD along with the teams, the System Architect/Engineer decomposes the solution into subsystem/components and identifies targets for each. The teams then explore multiple alternative concepts for each subsystem, filtering out options that either 1) provide less economic value, or 2) are flawed in some way that cannot meet the targets or violate laws of physics, etc. [Ward 1]

In the Scaled Agile Framework® (SAFe®), teams explore set-based design alternatives, applying a hypothesis-driven Minimum Viable Product (MVP) approach with a Lean Startup mindset (see Continuous Delivery Pipeline). Design alternatives include hypotheses and assumptions. MVPs may also define experiments to gain the knowledge that allows teams to validate or invalidate those hypotheses, filter alternatives, and arrive at the optimal economic decision.

Figure 2 below provides an example of a significant learning Milestone (a deadline for the decision) in which designers of a future autonomous vehicle have to select the technology to support a new initiative that prevents forward collisions. Teams explore alternatives for a new ‘Obstacle Detection System (ODS)’ vehicle subsystem using lidar, radar, and camera technologies. With their corresponding hypothesis statement, they create Enablers that explore cost tradeoffs, support for environmental and weather conditions, impact on vehicle design and manufacturing, quality of detection, etc. Teams record the results in the Solution Intent and filter the design alternatives based on validated learning.

Figure 2. Example where a future autonomous vehicle has a significant learning milestone

Of course, maintaining more than one design option also comes at a cost—the cost to develop and maintain those options, even if they are mostly model or paper based. (Note: Reinertsen [3] points out that maintaining multiple design options is one form of U-curve optimization, and sometimes the optimum number on the curve is one!) But if there’s a high degree of innovation or variability, or immovable deadlines (’This crop combine must leave the factory in January.’), a set-based design may be the best choice. In this case, design efficiency depends on several factors:

  • Flexibility – preserving a broad set of design options for as long as possible
  • Cost – minimizing the cost of multiple options through modeling, simulation, and prototyping
  • Speed – facilitating learning through early and frequent validation of design alternatives

To achieve this efficiency, some recommended practices are described below.

Increase Flexibility in Interfaces and Design

Complex systems are built out of subsystems and component elements that collaborate to produce system behavior. Traditionally, specifications are defined early with minimal understanding for what is possible, and less concern for what the tradeoffs might be. A set-based approach makes decisions from tradeoffs based on validated learning. Final designs and specifications emerge from teams exploring alternatives and understanding design tradeoffs.

In SAFe, the System Architect/Engineer typically defines the connection point between subsystems and provides the context for teams to negotiate their own interfaces. In the example in Figure 2, the system has a new ODS component responsible for preventing forward collisions. But teams must determine their own interfaces between the ODS and other subsystems—such as chassis (sensor mounting), powertrain (deceleration), braking, lighting (brake lights), etc.

However, interfaces are not rigid specifications. Lean allows interfaces to vary at the points of subsystem intersection. At these intersection points, system engineers may specify ranges for negotiation: (‘What can you do for me with additional allocation of space, weight, or power?’) This approach allows systems engineers to manage the system-level allocations while teams make their own detailed decisions. This creates a collaborative environment for system-level learning, negotiation, and optimal ROI.

Modeling, Simulation, and Prototyping

The process of modeling, simulating, and prototyping allows early empirical system validation; and it provides the initial learning points that will help eliminate some design alternatives and confirm others. Model-Based Systems Engineering (MBSE) supports set-based design through a disciplined, comprehensive, and rigorous approach. It incorporates a broad spectrum of modeling techniques specific to the industry and product type, including design thinking and user-centered design. These techniques should be applied to the parts of the system where the risk is highest, significantly reducing the cost of maintaining design alternatives for a longer period of time.

Frequent Integration Points

During development periods where new designs are being explored, uncertainty abounds. True knowledge is scarce. The only way to resolve the uncertainty is to test the design through early and frequent integration of the system components. In SAFe, these integration points are driven in part by System Demos, which occur on a fixed two-week cadence, and by Solution Demos, which typically occur on the longer Program Increment (PI) cadence. In fact, without this frequent integration, SBD practices may create a false sense of security. They may even increase risk, as it’s possible none of the design alternatives will really meet the identified targets. Frequent integration supports empirical learning with new insights used to reduce options as the system evolves, as Figure 3 illustrates.

Figure 3. Frequent integration provides critical learning points that narrow design alternatives

Take a Systems View

On large systems, decisions may span many initiatives. For example, the ODS technology decision for the autonomous vehicle should consider more than the current initiative to avoid front collisions. System Architecture/Engineers own the technical vision and help ensure that the system meets future goals. As Figure 4 illustrates, they must guide teams’ understanding of the larger solution when making significant technology decisions.

Figure 4. Technology decisions must look beyond the current initiative

As mentioned above, set-based design has a cost. Exploring options can even contribute to Weighted Shortest Job First (WSJF) and the overall Cost of Delay (CoD). System architect/engineers must balance the possibility for over-engineering the solution (i.e.,YAGNI–You aren’t going need it.) with being unprepared for upcoming capabilities that may incur larger costs and rework ahead. And like every decision in SAFe, ‘how much’ exploration should be an economic decision.

Use Set-based Design in Fixed-Schedule Programs

Set-based design is particularly effective for programs that require a high degree of fixed schedule commitments. Even if some of the more reliable design options don’t provide the degree of innovation or enhanced performance that the systems developers would prefer, it makes sense to keep multiple design options present, since the schedule is unmovable. And when the deadline is unyielding, teams must do what they can within the schedule.

Adapting Planning

Explicit and regular planning provides the opportunity for evaluating different design alternatives and directly supports set-based thinking. PI Planning defines the overall intent for the PI, fostering alignment on the constraints and requirements that will govern design alternatives under consideration. Iteration Planning plays a more tactical role. It allows teams to adjust during PI execution as they learn more from frequently integrating and reviewing increments of value.

Economic Trade-offs in Set-based Design

Different design options have different economic implications. So understanding SBD requires knowledge of the macroeconomic goals and benefits of the system. One way to look at this is to place the alternatives on a spectrum, where a certain ‘weight’ can be associated with each option.

Some of the significant economical indicators may include:

  • Cost of development
  • Cost of manufacturing
  • Performance and reliability
  • Cost of support
  • Development time
  • Technical risks

These indicators help illustrate which design options provide the greatest benefits. For instance, in the earlier collision prevention example, the trade-off between the accuracy of various detection technologies and cost of manufacture can make a big difference, as Figure 5 demonstrates.

Figure 5. A trade-off curve between an economic indicator (Cost) and a performance requirement (Error Margin) provides guidance for choosing among set-based designs

In summary, up-front commitment to a specific, detailed design can rarely survive contact with empirical evidence. A proper understanding of the economic trade-offs and SBD provides an adaptive approach with a wider systems perspective, better economic choices, and more adaptability to existing constraints.


Learn More

[1] Ward, Allen, and Durward Sobek. Lean Process and Product Development. Second edition. Lean Enterprise Institute, Inc., 2014.

[2] Oosterwal, Dantar. The Lean Machine. Amacom, 2010.

[3] Reinertsen, Don. Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing, 2009.

 

Last update: 11 August, 2017