Assume variability; preserve options.

—SAFe Principle #3

 

Set-based Design

Set-based Design is a practice that keeps requirements and design options flexible for as long as possible during the development process. Instead of teams choosing a single “point” solution upfront, set-based design identifies and simultaneously explores multiple options and eliminate poorer choices over time. It enhances flexibility in the design process, commits to technical solutions only after validating assumptions, and produces better economic outcomes.

System development can be described as a process of continuously converting uncertainty to knowledge. No matter how well a system is initially defined and designed, real customer needs and technological choices are both uncertain and evolving. Therefore, the understanding of how a system needs to be implemented must adapt over time.

Set-based Design (SBD) maintains multiple requirements and design options for a longer period in the development cycle. As the timeline advances, set-based design uses empirical data to narrow the focus to the final design option. Through this approach system builders can assume variability and preserve options late into the game, providing maximum flexibility.

Details

Point-based design commits to a set of requirements and a single design strategy early in the process. It often leads to late discoveries that require substantive rework as the deadline approaches, necessitating shortcuts, quality compromises, and, worse, missed program commitments and deadlines. Set-based design (SBD) maintains multiple design options through a longer period in the development cycle and uses the Continuous Delivery Pipeline cadence to provide feedback and learning.

SBD is an important practice for economic efficiency in Lean product development. It is described further in references [1] and [2]. Figure 1 shows the conceptual difference between set- and point-based design approaches.

Figure 1. Point-based design commits to a design up front. Set-based design maintains multiple alternatives for a longer period.

Increase Economic Efficiency with Set-based Design

Using SBD, in conjunction with the teams, the System Architect/Engineer decomposes the solution into subsystem/components and identifies targets for each. Teams then explore multiple alternative concepts for each subsystem, filtering out options that either 1) provide less economic value or 2) are flawed in some way in that they cannot meet the targets, violate laws of physics, etc. [Ward 1].

Using a Lean Startup mindset (see Continuous Delivery Pipeline), SAFe teams explore set-based design alternatives using an hypothesis-driven Minimum Viable Product (MVP approach. Design alternatives include hypotheses and assumptions. MVPs may also define experiments to gain the knowledge that allow teams to validate or invalidate those hypotheses, filter alternatives, and arrive at the optimal economic decision.

Figure 2 below provides an example where a future autonomous vehicle has a significant learning Milestone ( a deadline for the decision) to select the technology to support a new initiative that prevents forward collisions. Teams explore alternatives for a new “Obstacle Detection System (ODS)” vehicle subsystem using lidar, radar, and camera technologies. They create Enablers with their corresponding hypothesis statement that explore tradeoffs in terms of cost, support for environmental conditions (weather), impact on vehicle design and manufacturing, quality of detection, etc. Teams record results in the Solution Intent and filter the design alternatives based on validated learning.

Figure 2. example where a future autonomous vehicle has a significant learning milestone

Of course, maintaining more than one design option also comes at a cost—the cost of developing and maintaining the options, even if they are mostly model or paper based. (Note: Reinertsen [3] points out that maintaining multiple design options is one form of U-curve optimization, and sometimes the optimum number on the curve is one!) But in the case of a high degree of innovation or variability, and in the case of immovable deadlines (“This crop combine must leave the factory in January”), a set-based design may be the best choice. In this case, design efficiency depends on several factors:

  • Flexibility – preserving a broad set of design options for as long as possible
  • Cost – minimizing the cost of multiple options through modeling, simulation, and prototyping
  • Speed – facilitating learning through early and frequent validation of design alternatives

To achieve this efficiency, some recommended practices are described below.

Increase Flexibility in Interfaces and Design

Complex systems are built out of subsystems and component elements, which collaborate to produce system behavior. Traditionally, specifications are defined early with minimal understanding for what was possible and less concern for what the tradeoffs could be. A set-based approach makes decisions from tradeoffs derived from validated learning. Final designs and specifications emerge from teams exploring alternatives and understanding design tradeoffs.

In SAFe, the System Architect/Engineer typically defines the connection point between subsystems and provides the context for teams to negotiate their own interfaces. From the example in Figure 2, the system has a new ODS component responsible for preventing forward collisions. But teams must determine their own interfaces between the ODS and other subsystems such as chassis (sensor mounting), powertrain (deceleration), braking, lighting (brake lights), etc.

Interfaces are not rigid specification however. Lean allows interface to vary at the points of subsystem intersection. At these intersection points, system engineers may specify ranges for negotiation – “what can you do for me with additional allocation of space, weight, or power”. This approach allows systems engineers to manage the system-level allocations while teams make their own detailed decisions and creates an environment for system-level learning, negotiation, and optimal economics.

Modeling, Simulation, and Prototyping

The process of modeling, simulating, and prototyping allows for empirical system validation early, and it provides early learning points that will help eliminate some design alternatives and validate others. Model-Based Systems Engineering supports of set-based design via a disciplined, comprehensive, and rigorous approach to modeling. It incorporates a broad spectrum of modeling techniques specific to the industry and product type, including design thinking and user centered design. These techniques should be applied to the aspects of the system where the highest risk is likely to occur, thereby significantly reducing the cost of maintaining design alternatives for a longer period of time.

Frequent Integration Points

During development periods where new designs are in flight, uncertainty abounds and true knowledge is scarce. The only way to resolve the uncertainty is to test the design via early and frequent integration of the system components. In SAFe, these integration points are driven in part by System Demos, which occur on a fixed two-week cadence, and by Solution Demos, which typically occur on the longer Program Increment (PI) cadence. In fact, without this frequent integration, SBD practices may create a false sense of security and thereby even increase risk, as perhaps none of the design alternatives will really meet the identified targets. Frequent integration supports this empirical learning with new knowledge that is used to reduce options as the system evolves, as Figure 3 illustrates.

Figure 3. Frequent integration provides critical learning points that narrow design alternatives

Take a Systems View

On large systems, decisions may span many initiatives. For example, the obstacle detection technology decision for the autonomous should consider more than the current initiative to avoid front collisions. System Architecture/Engineers own the technical vision and help ensure the system meets future goals. As Figure 4 illustrates, they must guide teams’ understanding of the larger solution when making significant technology decisions.

Figure 4. Technology decisions must look beyond the current initiative

As mentioned above, set-based design has a cost. Exploring options can even contributes to WSJF and the overall cost of delay. System architect/engineers must balance the possibility for over-engineering the solution (YAGNI – you aren’t going need it) with being unprepared for future capabilities that may incur larger future costs and require rework. And like every decision in SAFe, ‘how much’ exploration should be an economic decision.

Use Set-based Design in Fixed-Schedule Programs

Set-based design is particularly effective in programs that require a high degree of fixed schedule commitments. After all, since the schedule is unmovable, it makes sense to keep multiple design options present, even if some of the more reliable design options do not necessarily provide the degree of innovation or enhanced performance that the systems developers would otherwise prefer. But when the deadline is sacrosanct, teams must do what they can within the schedule.

Adapting Planning

Explicit and regular planning provides the opportunity for evaluating different design alternatives and directly supports set-based thinking. PI Planning defines the overall intent for the PI and fosters alignment on the constraints and requirements that will govern the design alternatives under consideration. Iteration Planning plays a more tactical role, allowing teams to adjust during PI execution as they learn more from frequently integrating and reviewing increments of value.

Economic Trade-offs in Set-based Design

Different design options have different economic implications, so understanding SBD requires an understanding of the macroeconomic goals and benefits of the system. One way to look at this is to place the alternatives on a spectrum, where a certain “weight” can be associated with each option.

Some of the economically significant indicators may include cost of development, cost of manufacturing, performance and reliability, cost of support, development time, and technical risks. These indicators help illustrate which design options provide the greater benefit. In the earlier collision prevention example, for instance, the trade-off between the accuracy of various detection technologies and the cost of manufacture can make a big difference, as Figure 5 demonstrates.

Figure 5. A trade-off curve between an economic indicator (Cost) and a performance requirement (Error Margin) provides guidance for choosing among set-based designs

In summary, up-front commitment to a specific, detailed design can rarely survive contact with empirical evidence. With a proper understanding of the economic trade-offs, set-based design provides an adaptive approach with a wider systems perspective and provides for better economic choices and more adaptability to existing constraints.

 

Learn More

[1] Ward, Allen, and Durward Sobek. Lean Process and Product Development. Second edition. Lean Enterprise Institute, Inc., 2014.

[2] Oosterwal, Dantar. The Lean Machine. Amacom, 2010.

[3] Reinertsen, Don. Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing, 2009.

 

Last update: 17 June, 2017