Trust, but verify.
—Ronald Reagan, citing a Russian proverb
Compliance refers to a strategy and a set of activities and artifacts that allow teams to apply Lean-Agile development methods to build systems that have the highest possible quality, while simultaneously assuring they meet any regulatory, industry, or other relevant standards.
Note: This article is supported by a white paper, ‘Achieving Regulatory and Industry Standards Compliance with the Scaled Agile Framework®(SAFe®),’ which you can download here. In addition, there is a SAFe webinar on the topic, with Q&A, which can be viewed here.
Enterprises use SAFe to build some of the world’s largest and most important systems, many of which have unacceptable social or economic costs of failure. Examples of these high-assurance systems include medical devices, automobiles, avionics, banking and financial services, and aerospace and defense. To protect public safety, these systems are often subject to extensive regulatory or customer oversight and rigorous compliance requirements. In addition, many enterprises are subject to other government regulations (examples: Sarbanes-Oxley, HIPAA, ACA, state insurance regulations) that require similar attention and audits to ensure compliance.
Historically, organizations operating under such regulations have relied on comprehensive quality management systems (QMS). Legacy QMS systems are based on traditional, phase-gated development models and struggle to keep pace with accelerating time-to-market demands and Lean-Agile practices. Of greater concern is that even when the higher Cost of Delay (CoD) is accepted, these traditional approaches often do not increase quality or eliminate risk. As Deming notes, “Inspection is too late. The quality good, or bad, is already in the product.”
Continually addressing compliance concerns is one of the nine practices of SAFe’s Enterprise Solution Delivery competency. This article extends that knowledge and offers guidance on how to apply Lean-Agile methods to build these systems faster and better, while also addressing critical compliance requirements.
Traditional waterfall practices often mandate that full system specifications are defined and committed to in detail, up-front, long before all the real system behaviors can be known. Worse, the sequential nature of phase-gate development produces large batches of work, long cycles between system integration points, and late feedback. In addition, compliance activities are typically deferred until the end of the project, providing little insight into compliance progress.
This often results in missed deadlines, disappointing business or mission outcomes, lower quality, and substantial (and late) compliance challenges. In contrast, high-assurance Lean-Agile development builds in quality incrementally—early and throughout the development lifecycle.
The Role of the Quality Management System
To satisfy compliance requirements, organizations must demonstrate that their system meets its intended purpose and has no unintended consequences that might cause harm. They must also develop the objective evidence required to prove that the system conforms to those standards. To that end, organizations that build high-assurance systems define their approved practices, policies, and procedures in a QMS. These systems ensure that development activities and outcomes comply with all relevant regulations and provide the required documentation to prove it.
Unfortunately, many QMS systems are heavily influenced by traditional phase-gated waterfall methods. This seriously inhibits, and can even prevent, the adoption of newer methods, as the older methods are hard-coded into the only approved way of working. As Figure 1 illustrates, SAFe describes an incremental approach to both development and compliance.
This means those who want the benefits of Lean-Agile development (faster time to market and higher quality, to name a few) will typically have to evolve a Lean QMS.
The remainder of this article provides guidance on Lean-Agile strategies and patterns that develop these high-assurance systems.
Build the Solution and Compliance Incrementally
Even with a set of robust specifications, engineering teams never have all the answers when development begins. Instead, they have a set of hypotheses that must be tested through a series of short, iterative experiments, which provide validated learning to ideally advance toward the ultimate solution. Figure 2 highlights SAFe’s incremental development approach, comparing Shewhart/Deming’s Plan-Do-Check-Adjust (PDCA) learning cycles with a traditional waterfall model.
Figure 2 illustrates two important implications for compliance. First, building smaller, working parts of the solution early allows compliance activities to also begin early, removing the large bow wave of performing such actions at the end. Each increment assesses both the viability of the current solution and its progress toward compliance, providing early feedback on the system’s ultimate fitness for use. Second, specifications are created early and evolved over time in small batches, with faster feedback on decisions and the opportunity for continuous review and assessment.
Organize for Value and Compliance
Agile Release Trains (ARTs) are the primary value delivery organizations in SAFe. Each train requires all the skills necessary to build and release the Solution, including those responsible for Quality Assurance (QA), security, testing, and Verification and Validation (V&V). (Note: While some regulations require independent, objective assurance, compliance representatives can still participate continuously as members of the ART). The result is an ART designed as illustrated in Figure 3.
Solution and Product Management ensure that the Solution Intent and backlog properly reflect compliance requirements. Teams also ensure that their work includes appropriate compliance activities.
Build In Quality and Compliance
Built-In Quality is one of SAFe’s four Core Values, a dimension of SAFe’s Team and Technical Agility competency, and a core principle of the Lean-Agile Mindset. SAFe describes the use of Built-In Quality practices, including automation to detect compliance and quality problems and, when detected, stopping the entire system to focus everyone on resolving the problem. This philosophy applies systems thinking by ‘optimizing the whole,’ ensuring fast flow across the entire value stream and making quality everyone’s job. Quality is a culture, not a job title.
To that end, compliance concerns are also built directly into the development process, and automated wherever possible, as illustrated in Figure 4.
Not all compliance activities can be automated, however, as some regulatory requirements mandate manual activities, including activities like Failure Mode and Effects Analysis (FMEA) and audits. This work is simply planned as part of the team backlog. The goal is to conduct these activities and reviews as the solution is being built, reducing the last sign-off activity from a large, extended event to a quick and boring ‘non-event.’
This way, the program receives early feedback on the degree to which the team’s compliance activities are being met and, conversely, how those activities may be impacting team performance. Figure 5 shows the feedback cycle between team activities and the practices defined by the Lean QMS.
Continuously Verify and Validate (V&V)
Most high-assurance systems require V&V to ensure that:
- The system works as designed (verification)
- It meets the needs of the user (validation)
V&V must always occur against a known set of requirements. Otherwise, there is nothing to V&V against. As Figure 6 illustrates, SAFe uses solution intent as the repository for the existing and emerging requirements and designs.
Traceability within the solution intent ensures that the artifacts produced each Program Increment (PI)—the actual software, hardware components, etc.—address regulatory and compliance specifications, providing end-to-end evidence that V&V requirements have been met.
The SAFe Requirements Model Supports V&V
In SAFe, all elements of the requirements have test cases (see Figure 7), which are created at the same time as the functionality.
Each increment yields new functionality and, consequently, adds new tests. As the number of tests grows, automation is vital to prevent testing activities from becoming a bottleneck.
Make V&V and Compliance Activities Part of Regular Flow
Verification and validation activities are performed in small batches. Each work item ensures it includes all the necessary information required for compliance. Any reviews, audits, and sign-offs are included in the work item’s Definition of Done (DoD) as shown in Figure 8. Each PI, the System Demo evaluates the solution’s progress towards its operational goals and the objective evidence necessary to meet compliance.
Release Validated Solutions on Demand
- Validation testing of the final release candidate (examples: medical trial, flight test)
- Review of the objective evidence required before production approval and release
- Customer, regulatory, UAT, signoffs, document submissions
Even then, Lean-thinking organizations always strive to fully automate delivery and, where possible, build in automated final release checks as part of a SAFe Continuous Delivery Pipeline and release on demand.
Learn More Leffingwell, Dean. Agile Software Requirements. Pearson Education. 2011.
Last update: 9 December 2019