Enterprise

• Business Agility Self-Assessment

Portfolio

• Lean Portfolio Metrics
• Value Stream Key Performance Indicators
• Lean Portfolio Management Self-Assessment
• Continuous Learning Culture Self-Assessment
• Organizational Agility Self-Assessment

Large Solution

• Solution Predictability Measure
• Solution Performance Metrics
• Enterprise Solution Delivery Self-Assessment

Essential

• Feature Progress Report
• Program Predictability Measure
• Performance Metrics
• Cumulative Flow Diagram
• Continuous Delivery Pipeline Efficiency
• Deployments and Releases per Timebox
• Recovery over Time
• Innovation Accounting and Leading Indicator
• DevOps Health Radar
• Iteration Metrics
• Team PI Performance Report
• Lean Agile-Leadership Self-Assessment
• Agile Product Delivery Self-Assessment
• Team and Technical Agility Self-Assessment

Enterprise Metrics

Business Agility Self-Assessment

The SAFe business agility self-assessment is a tool that helps portfolios evaluate their progress toward business agility. While the Implementation Roadmap provides a script for organizations implementing SAFe, the business agility self-assessment provides multiple landmarks along this journey for easier navigation.

The assessment is built around the seven core competencies of business agility. Each competency then splits into three dimensions, and the assessment provides a set of statements that determine progress along the 21 dimensions shown below in Figure 1.

The Measure and Grow article outlines the recommended steps for facilitating the business agility self-assessment, analyzing the results and identifying opportunities for growth.

Portfolio Metrics

Lean Portfolio Metrics

Figure 2 provides a comprehensive set of Lean Portfolio Management metrics that can be used to assess internal and external progress for an entire portfolio.

Lean Portfolio Management Self-Assessment

The Lean Portfolio Management competency aligns strategy and execution by applying Lean and systems thinking approaches to strategy and investment funding, Agile portfolio operations, and governance.

The Lean portfolio management self-assessment, Figure 3, enables the enterprise to assess their proficiency in this competency and identify potential areas for improvement. In support of this, the Lean Portfolio Management Grows are a set of recommended resources and actions organizations can leverage to create a learning organization that innovates and relentlessly improves.

Continuous Learning Culture Self-Assessment

The Continuous Learning Culture competency describes a set of values and practices that encourage individuals—and the enterprise as a whole—to continually increase knowledge, competence, performance, and innovation. This is achieved by becoming a learning organization, committing to relentless improvement, and promoting a culture of innovation.

The continuous learning culture self-assessment, Figure 4, enables the enterprise to assess their proficiency in this competency and identify potential areas for improvement. In support of this, the Continuous Learning Culture Grows are a set of recommended resources and actions organizations can leverage to create a learning organization that innovates and relentlessly improves.

Organizational Agility Self-Assessment

The Organizational Agility competency describes how Lean-thinking people and Agile teams optimize their business processes, evolve strategy with clear and decisive new commitments, and quickly adapt the organization as needed to capitalize on new opportunities.

The organizational agility self-assessment, Figure 5, enables the enterprise to assess their proficiency in this competency and identify potential areas for improvement. In support of this, the Organization Agility Grows are a set of recommended resources and actions organizations can leverage to create a learning organization that innovates and relentlessly improves.

Large Solution Metrics

Solution Train Predictability Measure

The Agile Release Trains (ARTs) predictability measures are summarized to calculate the Solution Train’s predictability measure, as illustrated in Figure 6.

Solution Train Performance Metrics

The ARTs performance metrics are summarized to calculate the Solution Train’s performance metrics, as shown in Figure 7.

Enterprise Solution Delivery Self-Assessment

Enterprise Solution Delivery describes how to apply Lean-Agile principles and practices to the specification, development, deployment, operation, and evolution of the world’s largest and most sophisticated software applications, networks, and cyber-physical systems.

The enterprise solution delivery self-assessment, Figure 8, enables the enterprise to assess their proficiency in this competency and identify potential areas for improvement. In support of this, the Enterprise Solution Delivery Grows are a set of recommended resources and actions organizations can leverage to create a learning organization that innovates and relentlessly improves.

Essential Metrics

Feature Progress Report

The feature progress report tracks the status of features and enablers during PI execution. It indicates which features are on track or behind at any point in time. The chart has two bars:

• Plan – Represents the total number of stories planned.
• Actual – Represents the number of stories completed. The bar is shaded red or green, depending on whether or not the item is on track.

Figure 9 gives an example of a feature progress report.

Program Predictability Measure

The team PI performance reports are summarized to determine the program predictability measure, as illustrated in Figure 10. The report compares actual business value achieved to planned business value.

For more on this approach, see chapter 15 of Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise [1].

Program Performance Metrics

The end of each PI is a natural and significant measuring point. Figure 11 shows a set of performance metrics for a program.

Cumulative Flow Diagram

The Cumulative Flow Diagram (CFD) is made up of a series of lines or areas that show the amount of work in different Kanban states. For example, the typical states of the program Kanban are:

• Funnel
• Analyzing
• Backlog
• Implementing
• Validating on staging
• Deploying to production
• Releasing
• Done

Figure 12 shows the number of features in each Kanban state by day. The thicker areas in the CFD represent potential bottlenecks.

Continuous Delivery Pipeline Efficiency

The pipeline efficiency compares the amount of touch time versus wait time. Some of the information can be sourced automatically from tools, especially Continuous Integration, and Continuous Deployment, while other data requires manually recording in a spreadsheet. The value stream mapping technique is often applied to analyze problems identified in this report.

Note: The touch time represents when the team is adding value. Typically, touch time is only a small proportion of the total production time, most of the time is spent waiting, such as when moving work, waiting in queues and so on. This is clearly shown in Figure 13.

Deployments and Releases per Timebox

This metric is meant to demonstrate whether the program is making progress toward deploying and releasing more frequently. It can be viewed on a PI basis, as shown in Figure 14.

Or we can zoom in to see how releases are handled in mid-PI, as shown in Figure 15.

Recovery over Time

This report measures the number of rollbacks that occurred either physically or by turning off feature toggles. The date when a solution was deployed or released to production is also plotted here to determine if there is a relationship between the two (Figure 16).

Innovation Accounting and Leading Indicators

One of the goals of the Continuous Delivery Pipeline is to enable the organization to run experiments quickly to allow Customers to validate the hypotheses. As a result, both Minimal Marketable Features (MMFs) and Minimal Viable Products (MVPs) must define the leading indicators to measure progress toward the benefit hypothesis. Avoid relying on vanity metrics that do not measure real progress.

Figure 17 shows some metrics that were gathered from the SAFe website to demonstrate leading indicators for our development efforts.

DevOps Health Radar

The DevOps Health Radar is a tool to assess the progress of your program in improving the flow of value through Continuous Delivery Pipeline. Figure 18 shows the 16 sub-dimensions that programs should use to assess their maturity. It helps to identify the sub-dimensions in which we are sitting, crawling, walking, running, or flying, and identify places to improve.

You can also go to our partner’s site, AgilityHealth, to take an online version of the assessment at https://agilityhealthradar.com/safe-devops-assessment.

Iteration Metrics

Each Agile Team gathers the iteration metrics they’ve agreed to collect. This occurs in the quantitative part of the team retrospective. Figure 19 illustrates the chart for the measurements of one team.

Team PI Performance Report

During the PI System Demo, the Business Owners, Customers, Agile teams, and other key stakeholders rate the actual business value (BV) achieved for each team’s PI Objectives as shown in Figure 20.

Reliable trains should operate in the 80 – 100 percent range; this allows the business and its outside stakeholders to plan effectively. Below are some notes about how the report works:

• The planned total BV does not include uncommitted objectives to help the reliability of the train
• The actual total BV does include uncommitted objectives
• The achievement percentage is the actual BV ÷ planned BV
• A team can achieve greater than 100 percent (as a result of uncommitted objectives achieved)

Individual team totals are rolled up into the program predictability measure (see Figure 20).

Lean-Agile Leadership Self-Assessment

The Lean-Agile Leadership competency describes how Lean-Agile Leaders drive and sustain organizational change and operational excellence by empowering individuals and teams to reach their highest potential. They do this through leading by example; learning and modeling SAFe’s Lean-Agile mindset, values, principles, and practices; and leading the change to a new way of working.

The Lean-Agile leadership self-assessment, Figure 21, enables the enterprise to assess their proficiency in this competency and identify potential areas for improvement. In support of this, the Lean-Agile Leadership Grows are a set of recommended resources and actions organizations can leverage to improve the effectiveness of leaders as exhibit the Lean-Agile mindset and guide the organization through the changes associated with SAFe implementation.

Agile Product Delivery Self-Assessment

Grounded in a clear understanding of the customer, the Agile Product Delivery competency leverages a design thinking mindset to an increasingly automated continuous delivery pipeline that provides the ability to release value as, and when, needed.

The Agile product delivery self-assessment, Figure 22, enables the enterprise to assess their proficiency in this competency and identify potential areas for improvement. In support of this, the Agile Product Delivery Grows are a set of recommended resources and actions organizations can leverage to improve the effectiveness of their product delivery competency.

Team and Technical Agility Self-Assessment

The Team and Technical Agility competency is the collection of foundational practices on which Agile development is based. Only by creating high-performing teams and ARTs, that apply built-in quality, can true value be delivered quickly and reliably to customers.

The team and technical agility self-assessment, Figure 23, enables the enterprise to assess their proficiency in this competency and identify potential areas for improvement. In support of this, the Team and Technical Agility Grows are a set of recommended resources and actions organizations can leverage to improve the effectiveness of their Agile teams and trains.

Learn More

[1] Leffingwell, Dean. Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise. Addison-Wesley, 2011.

[2] Leffingwell, Dean. Scaling Software Agility: Best Practices for Large Enterprises. Addison-Wesley, 2007.

[3] https://www.youtube.com/watch?v=VOjPpeBh40s

Last update: 30 June 2020