Program and Value Stream Backlog

The emphasis should be on why we do a job.

—W. Edwards Deming

Program and Value Stream Backlogs Abstract

The Program and Value Stream Backlogs are the definitive repositories for all the upcoming work anticipated to advance the Solution. The backlogs consist of upcoming Features (Program Backlog) and Capabilities (Value Stream Backlog) that are intended to address user needs and deliver business benefits, as well as Enablers that advance learning and build the Architectural Runway.

The items in the backlog arise from various stakeholders—Customers, Business Owners, Product Management, Product Owners, Architects, and more—but the management of the backlog is the responsibility of Product Management (Program Backlog) and Solution Management (Value Stream Backlog). Effectively identifying, refining, prioritizing, and sequencing the backlog items (using WSJF) is the key to the economic success of the solution.

Since the backlog contains both new business functionality and the enablement work necessary to extend the architectural runway, capacity allocation can be applied to help ensure near- and long-term velocity and quality.

Details

The Program and Value Stream Backlogs are the repositories for all the upcoming work that affects the behavior of the Solution. Product and Solution Management develop, maintain, and prioritize the program and value stream backlogs respectively. The backlogs are a short-term holding area for Features and Capabilities that have gone through the Program and Value Stream Kanbans and have been approved for implementation. Backlog items should be estimated in story points, as Figure 1 illustrates.

Figure 1. Exploded view of program backlog, with story point size estimates
Figure 1. Exploded view of program backlog, with story point size estimates

Refining the Backlog

The Value Streams (VSs) and Agile Release Trains (ARTs) run a steady 8 – 10 week Program Increment cadence of planning > execution > demo > inspect and adapt. This steady rhythm is the heartbeat that drives backlog readiness as well. Appearing at a Pre-PI Planning or a PI Planning without a well-elaborated backlog adds unacceptable risk to the upcoming PI. To this end, the time between PI planning events is a busy time for product and solution management, as they are constantly in the process of refining the backlogs in preparation for the next PI planning. Making this process visible, and achieving “backlog readiness” for the upcoming PI, are the primary purposes of the ART and solution Kanbans. Backlog refinement typically includes:

  • Reviewing and updating backlog item definition and developing acceptance criteria
  • Working with the teams to establish technical feasibility and scope estimates
  • Analyzing ways to split backlog items into smaller chunks of incremental value
  • Determining the Enablers spawned by new features and capabilities, and establishing their capacity allocation

Prioritizing the Backlogs

Prioritizing the backlogs is a key economic driver for the solution. To this end, product and solution management use the Weighted Shortest Job First prioritization method for job sequencing. To recap, WSJF ultimately translates to a simple formula:

figure

 

Applying the formula requires that the numerator items be ranked only on a relative basis to each other. The denominator, job size, can be either a relative measure or an estimate in story points. (Note: The actual denominator of WSJF is job duration, though we use job size as a proxy. This is because duration depends on who does the work and what capacity allocation they can give the new work; that can be almost impossible to determine in advance of allocating the work. Since large jobs typically take longer to do, job size is a reasonable first approximation.)

With respect to job size, the denominator should represent only the remaining work for any items already under way. Then, if the remaining work for the item is too large to justify further investment relative to other jobs, the item can be called “good enough,” and the teams can move on to other priorities. This implicitly implements Reinertsen’s [2] key economic Principle E17: The sunk cost principle: Do not consider money already spent.

Preparing for PI Planning

The week or two prior to PI planning is a very busy time. Product and solution management do final backlog preparation, update the vision briefings, and work with Product Owners to further socialize the backlog prior to the event. System and Solution Architects/Engineering update enabler definitions and models and often develop use cases that illustrate how the features and capabilities work together to deliver the end user value.

Optimizing Value and Solution Integrity with Capacity Allocation

One of the challenges every ART and value stream faces is how to balance the backlog of business features and capabilities with the need to continuously invest in the Architectural Runway, provide time for exploration of requirements and design for future PIs, and create prototypes and models to enhance visibility into the problem areas. In order to avoid velocity reduction and to defer the need for wholesale replacement of components due to technological obsolescence, ARTs must invest continuously in implementing the enablers of the solution. This complicates the challenge of prioritizing work since different people can pull the teams in different directions, as Figure 2 shows.

Figure 2. Business vs. enabler backlog items dilemma
Figure 2. Business vs. enabler backlog items dilemma

To address this problem, teams apply capacity allocation, whereby they make a decision about how much of the total effort can be applied for each type of activity for an upcoming PI. Further, they establish an agreement to determine how the work is performed for each activity type. Examples of the results are given in Figure 3 and Table 1.

Figure 3. Capacity allocation for architecture in an ART backlog for a single PI
Figure 3. Capacity allocation for architecture in an ART backlog for a single PI

 

  1. At each PI boundary, we agree on the percentage of resources to be devoted to new features or capabilities vs. enablers.
  2. We agree that System and Solution Architects and Engineering have authority to prioritize enabler work.
  3. We agree that Product and Solution Management have authority to prioritize business backlog items.
  4. We agree to jointly prioritize our work based on economics.
  5. We agree to collaborate so as to sequence work in a way that maximizes Customer value.

Table 1: Sample policies for managing enabler and feature capacity allocation

While the agreed-to policies can persist for some time, the amount of capacity allocated should vary over time based on the context. In the context of an ART, this decision can be revisited as part of backlog refinement in preparation for each PI planning, while solution management and solution architect/engineering make similar decisions for the value stream as a whole before solution PI planning.

On Backlogs, Queues, Little’s Law, and Wait Times

It’s important to take a brief aside and discuss the relationship between backlog, wait times, and flow. The principle Manage queue length discusses the relationship in detail; refer to that for more complete understanding. However, it’s important to summarize that discussion here, because the program and value stream backlogs are the backlogs that can have the biggest impact on delivery time and throughput. Here’s a summary:

  1. Little’s Law illustrates that the average wait time for an item in a queue is equal to the average length of the queue divided by the average processing rate for an item in a queue. The longer the queue, the higher the wait time, and the higher the variability. (Think of the line at Starbucks: If the ten people ahead of you each order a tall coffee, you are going to be out of there in minutes; if they all order an extra-hot vanilla latte and a heated bagel, you might be late for your meeting, and it is not under your control.)
  2. Long queues are all bad, causing decreased motivation, poor quality, longer cycle times, higher variability (think Starbucks), and increased risk [2].
  3. Your program and value stream backlogs are not queues, as items can leapfrog others for faster delivery, and you can always choose not to service everything in the backlog. (Note that neither of these work at Starbucks.)
  4. However, if all the items in your backlog are committed to stakeholders, then your backlog behaves like a queue, and the longer it is, the longer your stakeholders are going to have to wait for service. And if they have to wait too long, they will find another coffee shop, as your shop just can’t meet their rapidly changing market needs.

Therefore, in order for a development program to be fast and responsive, teams must actively manage the backlogs and keep them short. Teams also must limit commitment to longer-term work, because some other item may come along that’s more important than a prior commitment. If a team has too many fixed and committed requirements in the backlog, they cannot respond quickly, no matter how efficient they are. Teams can be both reliable and fast only if they actively manage the backlog and keep it short.


Learn More

[1] Leffingwell, Dean. Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise. Addison-Wesley, 2011.

[2] Reinertsen, Don. Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing, 2009.

Last update: 20 April 2016