Trust, but verify.
—Ronald Reagan, citing a Russian proverbhttp://editor.scaledagileframework.com/compliance/
Compliance refers to a strategy and a set of activities and artifacts that allow teams to apply Lean-Agile development methods to build systems that have the highest possible quality, while simultaneously assuring they meet any regulatory, industry, or other relevant standards.
Note: This article is supported by a white paper, ‘Achieving Regulatory and Industry Standards Compliance with the Scaled Agile Framework®(SAFe®),’ which you can download here. In addition, there is a SAFe webinar on the topic, with Q&A, which can be viewed here.
Enterprises use SAFe to build some of the world’s largest and most important systems, many of which have unacceptable social or economic costs of failure. Examples of these high-assurance systems include medical devices, automobiles, avionics, banking and financial services, and aerospace and defense. To protect the public safety, these systems are often subject to extensive regulatory or customer oversight and rigorous compliance requirements. In addition, many enterprises are subject to other government regulations (examples: Sarbanes-Oxley, HIPAA, ACA, state insurance regulations) that require similar attention and audits to ensure compliance.
Historically, organizations operating under such regulations have relied on comprehensive quality management systems (QMS). Based on phase-gated development models, they’re intended to reduce risk and ensure compliance. But these traditional approaches don’t scale, even when some teams follow Agile practices. Nor do they keep pace with accelerating time-to-market demands. Of greater concern is that even when the higher Cost of Delay (CoD) is accepted, these traditional approaches often do not increase quality or eliminate risk. As Deming notes, “Inspection is too late. The quality good, or bad, is already in the product.”
This article offers guidance on how to apply Lean-Agile methods to build these systems faster and better, while also addressing critical compliance requirements.
Traditional waterfall practices often mandate that full system specifications are defined and committed to in detail, up-front, long before all the real system behaviors can be known. Worse, the sequential nature of phase-gated development produces large batches of work, long cycles between system integration points, and late feedback. In addition, compliance activities are typically deferred until the end of the project, providing little insight into compliance progress.
This often results in missed deadlines, disappointing business or mission outcomes, lower quality, and substantial (and late) compliance challenges. In contrast, high-assurance Lean-Agile development builds in quality incrementally—early and throughout the development lifecycle. And it does so while including the very elements and activities necessary to ensure compliance.
The Role of the Quality Management System
To satisfy compliance requirements, organizations must demonstrate that their system meets its intended purpose and has no unintended consequences that might cause harm. They must also develop the objective evidence required to prove that the system conforms to those standards. To that end, organizations that build high-assurance systems define their approved practices, policies, and procedures in a QMS. These systems are intended to ensure that development activities and outcomes comply with all relevant regulations and provide the required documentation to prove it.
Unfortunately, the QMS systems of many organizations are still heavily influenced by traditional phase-gated waterfall methods. This seriously inhibits, and can even prevent, the adoption of newer methods, as the older methods are hard-coded into the only approved way of working. As Figure 1 illustrates, SAFe describes an incremental approach to both development and compliance.
This means those who want the benefits of Lean-Agile development (faster time to market and higher quality, to name a few) will typically have to evolve a Lean QMS.
The remainder of this article provides guidance on Lean-Agile strategies and patterns that develop these high-assurance systems.
Build the Solution and Compliance Incrementally
Even with a set of robust specifications, engineering teams never have all the answers when development begins. Instead, they have a set of hypotheses that must be tested through a series of short, iterative experiments, which provide validated learning to ideally advance toward the ultimate solution. Figure 2 highlights SAFe’s incremental development approach, comparing Shewhart/Deming’s Plan-Do-Check-Adjust (PDCA) learning cycles with a traditional waterfall model.
Figure 2 illustrates two important implications for compliance. First, building smaller, working parts of the solution early allows compliance activities to also begin early, removing the large bow wave of performing such actions at the end. Each increment assesses the viability of the current solution, as well as progress toward compliance, providing early feedback on the system’s ultimate fitness for use. Second, specifications are created and evolved over time in small batches, with faster feedback on decisions and the opportunity for continuous review and assessment.
Organize for Value and Compliance
Agile Release Trains (ARTs) are the primary value delivery organizations in SAFe. Each train requires all the skills necessary to build and release the Solution, including those responsible for Quality Assurance (QA), security, testing, and Verification and Validation (V&V). (Note: While some regulations require independent, objective assurance, compliance representatives can still participate continuously as members of the ART). The result is an ART designed as illustrated in Figure 3.
Solution and Product Management ensure that the Solution Intent and backlog properly reflect compliance requirements. Teams also ensure that their work includes appropriate compliance activities.
Build In Quality and Compliance
Built-In Quality is one of SAFe’s four Core Values and a core principle of the Lean-Agile Mindset. SAFe describes the use of built-in quality practices, including automation to detect compliance and quality problems and, when detected, stopping the entire system to focus everyone on resolving the problem. This philosophy applies systems thinking by ‘optimizing the whole,’ ensuring fast flow across the entire value stream and making quality everyone’s job. Quality is a culture, not a job title.
To that end, compliance concerns are also built directly into the development process, and automated wherever possible, as illustrated in Figure 4.
Not all compliance activities can be automated, however, as some regulatory requirements mandate manual reviews, including activities like Failure Mode and Effects Analysis (FMEA) and audits. These are simply planned as part of the team backlog. The goal is to conduct these reviews as the solution is being built, reducing the last sign-off activity from a large, extended event to a quick and boring ‘non-event.’
This way, the program receives fast feedback on the degree to which the team’s compliance activities are being met and, conversely, how those activities may be impacting team performance. Figure 5 shows the feedback cycle between team activities and the practices defined by the Lean QMS.
Continuously Verify and Validate
Most high-assurance systems require V&V to ensure that:
- The system works as designed (verification)
- It meets the needs of the user (validation)
V&V must always occur against a known set of requirements. Otherwise, there is nothing to V&V against. As Figure 6 illustrates, SAFe uses solution intent as the repository for existing and emerging requirements and designs.
Traceability from solution intent ensures that the artifacts produced each Program Increment (PI)—the actual software, hardware components, etc.—address regulatory and compliance specifications, providing end-to-end evidence that V&V requirements have been met.
The SAFe Requirements Model Supports V&V
In SAFe, all elements of the requirements have test cases (see Figure 7), which are created at the same time as the functionality.
Each increment yields new functionality and, consequently, adds new tests. As the number of tests grows, automation is vital to prevent testing activities from becoming bottlenecks.
Make V&V and Compliance Activities Part of Regular Flow
By incrementally building the necessary artifacts in the solution content over a series of PIs, SAFe supports continuous verification, as Figure 8 illustrates.
Verification activities are implemented as part of the flow of value delivery (for example, backlog items or Definition of Done [DoD] as described above). While the artifacts will satisfy the objective evidence needed at the end of development, they are created iteratively throughout the life cycle. Validation occurs when Product Owners, Customers, and end users participate in ART planning, demos, and validation of fitness for purpose.
In the example in Figure 8, regulations require design reviews and that all actions be recorded and resolved. The “design review” Enabler backlog item offers evidence of the review, and its DoD ensures that actions are recorded and resolved per the Lean QMS. If needed, the actions themselves could be tracked as enabler stories. Regulations may also require that all changes be reviewed, which is addressed by a compulsory peer review DoD for all stories.
SAFe recommends building the integrated solution frequently (or elements of the system for cyber-physical solutions), at least at every Iteration for the System Demo. Building and integrating frequently allows continuous validation from User Acceptance Test (UAT), customers, and end users. Each iteration, the system demo provides objective evidence that the integrations perform as intended and that the entire system has advanced forward while maintaining quality and compliance standards.
Release Validated Solutions on Demand
Finally, SAFe recognizes that although the product development process happens in a predictable cadence (Develop on Cadence), the release process (Release on Demand) may require additional activities. These can include:
- Validation testing of the final release candidate (examples: medical trial, flight test)
- Review of the objective evidence required before production approval and release
- Customer, regulatory, UAT, signoffs, document submissions
Even then, Lean-thinking organizations always strive to fully automate delivery and, where possible, build in automated final release checks as part of a SAFe Continuous Delivery Pipeline and release on demand.
Learn More Leffingwell, Dean. Agile Software Requirements. Pearson Education. 2011.
Last update: 16 April, 2018