Is agile the magic bullet for government project success? For happy implementations of enterprise software in the public sector? We’re agile advocates because of improved success rates using agile product development, agile project management, and agile country development. Our methodology, A-i3+qM is built on proven good practices and over 35 years of exclusive government implementations.
What about organization change management?
Agile techniques, when adapted to government, integrate change. Beyond software change management to true organizational change. Government Resource Planning (GRP) software configured to meet real requirements. And, effectively in use.
When properly practiced, agile engages stakeholders early and often. Change resistance dissipates as issues addressed and progress proven. As Jason Little observes, “Agile isn’t about “Agile”, it’s about change“.
Organizational change is particularly critical in the implementation of new government financial, human resources, procurement, or performance systems in government. In the Cynefin framework (diagram above), the implementation of GRP is beyond technical. It’s not simple, even though many would hope that governments follow “best practices”. Many of these practices are inappropriate for the government context at the time.
And, GRP is far more than an ICT project. It’s transformational. There’s legal reform, new processes, reorganization, and new performance measures.
The goal in GRP implementations is to support good practices, making implementations complicated. The reality is that implementations become complex, sometimes chaotic, because of unique public sector constraints:
- Stakeholders extend beyond government to citizens
- Public service capacity and incentives
- Politics, politics, and more politics
How is change integral in agile?
Although there are numerous agile techniques, (see glossary below), organizational change has been implicit since the creation of the Agile Manifesto. Manifesto values show how people, collaboration, and flexibility is paramount.
- Individuals and interactions over processes and tools: frequent interaction with stakeholders and users reduces change resistance while increasing likelihood that real needs addressed (An agile principle: “business people and developers must work together daily throughout the project.”)
- Working software over comprehensive documentation: documentation does not lead to shared understanding with stakeholders and users – frequently delays implementations giving time for change resistance to increase (An agile principle: “the most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”)
- Customer collaboration over contract negotiation: collaboration gives voice to stakeholders and users
- Responding to change over following a plan: context is learned through conversations, not through rigid planning
Agile can be done wrong. Agile operates well when trust is enabled and encouraged. Agile enables organizational change management when properly managed:
- Consideration: agile techniques like Design Thinking and PDIA encourages empathy and understanding problems from stakeholder and user perspectives
- IMPACT: stakeholders and users realize that systems aren’t imposed by outsiders
- Customer-Centric: agile works from the customer out (what we call “outside-in” in product management) rather than product in (what we call “inside-out” in product management)
- IMPACT: stakeholder and user fears and aspirations become baked into projects
- Communication: constant communication keeps stakeholders and users up-to-date through boards and other visual communications methods to demonstrate decisions, progress, and backlog as information radiator, a large display of critical team information that is continuously updated and located in a spot where the team can see it constantly
- IMPACT: stakeholders and users less prone to imagine the worst between milestones, where progress is visibly communicated
- Conversation: face-to-face conversations with stakeholders and users including workshops, brainstorming, and storyboards
- IMPACT: stakeholders and users recognize ability to adjust project deliverables while learning more about the project value to them, their organization, and their country to increase change acceptance
- Context: the country and government context becomes understood by project teams (and, FreeBalance has tools to enable this process)
- IMPACT: shared understanding among participants is enhanced to reduce change resistance
- Collaboration: project teams, stakeholders and users work together to set priorities, solve problems, and discuss solutions
- IMPACT: influence on the project through involvement increases change acceptance
- Creativity: leverage stakeholder and user creativity to better solve problems
- IMPACT: stakeholders help solve problems including change problems
- Confirmation: collaboration gives participants confirmation of project priorities throughout the project rather than sign-offs at major milestones as sole avenue
- IMPACT: project delays reduced because of iterative confirmation, giving little time for change resistance to build, while feedback loops validate stakeholder and user concerns
- Capacity: government project and governance domain capacity built during conversations and collaboration as mentoring, augmenting training and improving project success
- IMPACT: sustains organizational change trajectory beyond the project (and makes project more sustainable)
What is our experience with agile resistance?
Agile doesn’t compute doe many experts in the governance and PFM community. Lack of project success is often interpreted as lack of up-front design, and failure to execute on timelines. (My sense is that these are two contributors to failure.)
Agile sounds kind of new age wishy-washy to many. Unproven, theoretical. For example, why involve non-experts in solution design?
We encounter these objections frequently. Demonstration seems to be the best approach to communicate the impact of agile to improve decision-making, and organizational change management. In addition to using agile for implementations, we’ve used agile exercises at our annual FreeBalance International Steering Committee (FISC), change workshops, and country context exercises.
The change in attitude among public servants is palatable. At first, suspicion. Maybe a bit of confusion. Then, collaboration and conversation. It gets to the point where every working group wants to present findings to the workshop. Real collaboration.
Where isn’t organizational change management aligned with agile?
Orthodox organizational change management processes tend to be a linear sequenced way as described by the 5 I’s:
These are excellent frameworks for organizational change. The linear approach, however, seems to align with traditional waterfall approaches. My sense is that this is more a factor of working within the constraints of traditional project management. Just like agile can align with the CMMi and ISO standards, there’s no reason why these approaches cannot be leveraged iteratively.
Iteration may improve change outcomes, just as software development and project management success rates improve through agile.
We’ve seen some projects where organizational change management was considered an add-on to project management. This often results in “drive-by” and ineffective change management.
It makes sense to have change experts on projects. There is often a disconnection when change experts lack domain expertise. So, it’s more effective to have change integral to projects. To have project staff building change and credibility.
What is the organizational change management implication in agile projects?
Reed Deshler sees the need for a real-time approach to change management in agile projects. This includes adapting understood change management processes to be “fit for purpose” for individual projects. A study by Prosci analyzed change management practices in agile projects. The analysis found that successful change management in agile projects requires:
- Iterative approaches rather than linear
- OBSERVATION: all project components are iterative in agile, and the impact of change management is continuous to build change acceptance
- Change plans designed to adapt
- OBSERVATION: change plans should change as project participants learn more about context through conversation and collaboration – consider this as adapting to feedback looks
- Requires more upfront time
- OBSERVATION: agile provides the communications framework for change to facilitate planning, while more change planning comes through less project planning and documentation – where change management becomes a higher priority
Our experience with agile in the real world is that it generates the kind of quick wins needed to maintain change buy-in. Agile provides visible steps between the quick wins thanks to involvement, flexibility, and prototyping. We’ve benefited from the product design of the FreeBalance Accountability Suite thanks to configuration methods that enable confirming business rules and configuration in workshops. This enables the kind of agile development not available in fully customized government software or heavily customized private sector software like Enterprise Resource Planning (ERP).
What is the government context for change agents and change management?
As described earlier, the organizational change management context tends to be more complex in government than in business. Agile product development and project management methods do not directly address this complexity.
Problem-Driven Iterative Adaptation (PDIA), a public-sector and country development methodology, addresses this complexity challenge. PDIA was developed by the Building State Capacity facility of the Harvard Kennedy School of Government. This is much more than an academic exercise with numerous case studies in the real world. It’s part of the Doing Development Differently movement. We’re big fans of PDIA. Many of our executives have taken Executive Education for PDIA. Many of our staff have taken online PDIA courses.
What impresses me the most is that PDIA prioritizes change and persuasion. Stakeholder mapping, communications, and collaboration is built into PDIA. PDIA enables mapping roles and communications strategies.
|Substantive contributions||1. Construct, communicate problems
2. Come up with ideas for reform
3. Provide implementation view
|Procedural contributions|| 4. Provide formal authority
5. Motivate and inspire reform
6. Empower other agents
7. Conveners of small groups
8. Connectors to distributed agents
PDIA enables analyzing the change space: the overlap of authority, ability and acceptance. This is a particularly strong technique. As described in the Doing Development Differently Manifesto: success comes through legitimization “at all levels (political, managerial and social), building ownership and momentum throughout the process to be ‘locally owned’ in reality (not just on paper).”
PDIA recognizes that substantial change doesn’t depend on the heroic leader. Successful change in the public sector comes from many leaders. These are agents. This goes beyond the traditional approach of change agents to scaling agents. Scaled change is presented with a snowflake metaphor.
PDIA is becoming more accepted in the good governance and country development community. My hope is that PDIA doesn’t become a ceremony, when process becomes more important than outcomes.
What are the inhibitor to effective organization change using waterfall project management?
Many still believe that transformational projects, like implementing GRP, can be planned in much the same way as bridges or buildings. We see change resistance and four other major contributors to project success when comparing bridge building to software implementations:
- Anchoring: Software implementations are often anchored to the existing solution in place. Users want something familiar. However, existing systems are often fraught with inefficiencies and errors. Software is often customized to follow antiquated processes. Project decision-makers anchor to the “As-Is” processes. Unnecessary reports are created. Information quality often does not improve. The customizations lead to more errors. Of course, it’s all a bit odd when you consider that the purpose of getting a new system is because of deficiencies with the old.
- Documentation: Pages and pages of detailed documentation is created in traditional project management. Every stage requires documentation for sign-off by decision-makers. The more documentation, the more difficult it is for decision-makers to visualize what the eventual system will look like. For example, most methods for articulating specifications are very difficult for most people to understand. Easy for software engineers. This leads to delays. Many delays.
- Contracts: The desired outcome for systems integration firms is to deliver based on contractual obligations. Cost overruns and schedule changes are fine as long as the government will pay. Documentation, contracts, and contract amendments redefine success from original government objectives to contract compliance. That’s why systems integration firms get paid even when users are dissatisfied.
- Change Resistance: People see bridge construction in progress. Software is a different story. Many projects wait for full documentation sign-off before customization and configuration. This leads to increased change resistance. Users, government project team members, and decision-makers become disconnected from the implementation. The assumption is that the project is not going well because there is nothing to show for it.
- Team Bloat: The need for contract negotiators, documentation writers and explainers, leads to larger team sizes. This might seem like a good thing to those who haven’t read the Mythical Man Month, first published in the 1970s. Coordination and communications problems become exponential.
Waterfall can be successful in many circumstances, even with the risks. That’s why we developed a checklist.
Factors Favouring Waterfall A-F
Factors Favouring Agile G-Z
- Architectural Complexity: need to develop software architectures from scratch tend to require long design periods beyond multiple iterations – although this doesn’t apply to commercially-available architectures like the FreeBalance Accountability Platform
- Problems Well-Understood: deep insight into problems reduces the need to gather additional insight from agile – although, many (most?) large government IT projects are solution-focused that rarely address real problems without changes
- Experience in Similar Circumstances: government organization experience implementing similar initiatives makes projects more predictable – although, many government organizations assume vendor experience out of context such as vendor experience in other contexts
- Number of Personnel: agile operates well with small motivated teams who operate efficiently – although it should be noted that waterfall projects require more personnel for communications, coordination and documentation than agile
- Scope Focus: fully articulated project scope, and a focus on scope driving time and resources, lends itself to controls in waterfall that reduces “scope creep” – although, many projects fail because of unbending scope changes favouring contract provisions over real government needs
- Human Capacity: high government human capacity in technology, project management, and Public Financial Management (PFM) mitigates many of the risks associated with waterfall – although high capacity is often associated with overconfidence
- Project Complexity: agile helps decompose complexity and structure activities based on priorities through prototypes, user input, and design analysis
- Project Uncertainty: agile is ideal when there is high project uncertainty
- IT Failure Rates: agile provides success tools for IT teams who have experienced a history of underwhelming results
- Cross-Domain Functions: waterfall works best in silos by engaging domain specialists, and agile functions best when functions cross silos such as finance, procurement, payroll, and tax administration
- Outcome-Focused: projects conceived to have results benefit from agile processes because waterfall projects focus on contractual compliance
- Transformational Change: projects that follow modernization, reform, or re-organization benefit from agile processes that are more transparent, communicative, and flexible than waterfall
- Custom Development: expectation for custom development (such as custom development using a technical platform, custom development using a governance platform, or custom development using ERP) benefits from agile that identifies value for every custom element
- Current Legacy Technology Level: government organization that use ancient technology (mainframes, COBOL, 4GLs), or legacy technology (most proprietary ERP systems, including Tier 1), benefit from the fresh eyes associated with agile
- User Involvement: agile excels when users are required to articulate problems, identify solutions, test functionality, and championing new systems
- Functionality Change: agile excels when there are many new functions will be introduced compared to legacy system
- Greenfield Implementation: upgrades to existing systems can be implemented in waterfall, while net-new software implementation benefit from agile
- Requirements Change: waterfall expects very little change to requirements, while agile is resilient to change by identifying specification changes early to reduce costs compared to identifying needed changes later in projects
- System Dependencies: waterfall processes benefit from fewer system dependencies, such as integration points with other systems, while agile is better able to integrate across multiple government domains, and underlying technologies
- Product Novelty: agile is ideal when projects are implementing relatively new product suites
- Change Resistance: waterfall is not effective in situations where user and managerial resistance to change is high, while agile has more built-in processes that facilitate and integrate change management (bolt-on change management in waterfall projects is often ineffective)
- Quality Focus: waterfall processes put testing and quality assurance late in the methodology, while agile focuses on quality in each “user story” resulting in improving quality early
- System Extensibility: agile excels when existing or new software needs to extend to additional functions through reuse
- Custom Reports & Dashboards: the articulation of reports including replicating statutory reports in new software, and creating new reports, dashboards, and analysis benefits from agile iterations because users often recognize improvements once seeing report outputs
- Implementation Speed: agile excels in implementation speed, and provides more effective performance information (“cadence”) than waterfall, to predict project completion time
- Project Transparency: waterfall assumes separate teams who report around milestones, while agile assumes constant project transparency through Kanban, Scrum or Srumban boards, and frequent engagement with users and stakeholders
What is the FreeBalance agile methodology?
Our methodology has iterated over the years. The current version is A-i3+qM (accelerated, integrated, iterative, implementation-focused, quality management):
- Accelerated by eliminating as many legacy waterfall processes that lead to project problems. This includes unnecessary documentation and detailed project plans, in favour of workshops and short process steps. Team sizes are kept small to enable client communications and reduce coordination overhead.
- Integrated through a single methodology to support development and services implementation. This is integrated with customer requirements through the customer-centric processes. This provides transparency between the customer staff, the implementers, and the development team. Implementation and product development teams are integrated following DevOps practices.
- Iterative to be responsive to customer and implementation changes using phases. The methodology leverages the best of proven “lean” software development and services methodologies with workshops, short iterations, user stories, milestones and the ability to show progress. These techniques are extended beyond the development organization to implementation services leveraging productivity gains and ability to react to customer requirements.
- Implementation-focused with good practice templates and proven program management processes. This methodology is focused on the success of the customer implementation, rather than a software release that achieves internal or arbitrary goals. Implementation and product development are managed via a Program Management Office.
- Quality approach ensures that the software is released and supported meeting Commercial Off-the-Shelf (COTS) good practices with unit, system, stress and regression testing. Quality is integrated with implementation where FreeBalance tests based on customer environments.
A-i3+qM recognizes the deficiencies of traditional project management processes in GRP implementations. And, the patterns of failure.
All definitions, except for PDIA, via wikipedia.org
- Adaptive software development (ASD) is a software development process that grew out of the work by Jim Highsmith and Sam Bayer on rapid application development (RAD). It embodies the principle that continuous adaptation of the process to the work at hand is the normal state of affairs.
- Agile software development is an approach to software development under which requirements and solutions evolve through the collaborative effort of self-organizing and cross-functional teams and their customer(s)/end user(s). It advocates adaptive planning, evolutionary development, early delivery, and continual improvement, and it encourages rapid and flexible response to change.
- Design thinking refers to the cognitive, strategic and practical processes by which design concepts (proposals for new products, buildings, machines, etc.) are developed by designers and/or design teams. Many of the key concepts and aspects of design thinking have been identified through studies, across different design domains, of design cognition and design activity in both laboratory and natural contexts.
- DevOps is a set of software development practices that combines software development (Dev) and information technology operations (Ops) to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.
- Disciplined agile delivery (DAD) is a process decision framework that enables simplified process decisions around incremental and iterative solution delivery. DAD builds on the many practices espoused by advocates of agile software development, including Scrum, agile modelling, lean software development, and others.
- Dynamic systems development method (DSDM) is an agile project delivery framework, initially used as a software development method.
- Extreme programming (XP) is a software development methodology which is intended to improve software quality and responsiveness to changing customer requirements. As a type of agile software development, it advocates frequent “releases” in short development cycles, which is intended to improve productivity and introduce checkpoints at which new customer requirements can be adopted.
- Feature-driven development (FDD) is an iterative and incremental software development process. It is a lightweight or Agile method for developing software.
- Iterative and Incremental development is any combination of both iterative design or iterative method and incremental build model for development.
- Kanban (Japanese 看板, signboard or billboard) is a lean method to manage and improve work across human systems. This approach aims to manage work by balancing demands with available capacity, and by improving the handling of system-level bottlenecks.
- Lean software development is a translation of lean manufacturing principles and practices to the software development domain. Adapted from the Toyota Production System, it is emerging with the support of a pro-lean subculture within the Agile community. Lean offers a solid conceptual framework, values and principles, as well as good practices, derived from experience, that support agile organizations.
- Large-scale Scrum (LeSS) is a product development framework that extends Scrum with scaling rules and guidelines without losing the original purposes of Scrum.
- Model-driven engineering (MDE) is a software development methodology that focuses on creating and exploiting domain models, which are conceptual models of all the topics related to a specific problem. Hence, it highlights and aims at abstract representations of the knowledge and activities that govern a particular application domain, rather than the computing (i.e. algorithmic) concepts.
- Microsoft Solutions Framework (MSF) is a set of principles, models, disciplines, concepts, and guidelines for delivering information technology services from Microsoft. MSF is not limited to developing applications only; it is also applicable to other IT projects like deployment, networking or infrastructure projects. MSF does not force the developer to use a specific methodology (such as the waterfall model or agile software development).
- The Personal Software Process (PSP) is a structured software development process that is intended (planned) to help software engineers better understand and improve their performance by bringing discipline to the way they develop software and tracking their predicted and actual development of the code. It clearly shows developers how to manage the quality of their products, how to make a sound plan, and how to make commitments. It also offers them the data to justify their plans.
- Problem-Driven Iterative Adaptation (PDIA) is a a process of facilitated emergence which focuses on problems (not solutions) and follows a step by step process (not a rigid plan) that allows for flexible learning and adaptation, designed by the Building State Capacity facility of the Harvard Kennedy School of Government.
- Rapid-application development (RAD), also called Rapid-application building (RAB), is both a general term, used to refer to adaptive software development approaches, as well as the name for James Martin’s approach to rapid development. In general, RAD approaches to software development put less emphasis on planning and more emphasis on an adaptive process. Prototypes are often used in addition to or sometimes even in place of design specifications.
- The Rational Unified Process (RUP) is an iterative software development process framework created by the Rational Software Corporation, a division of IBM since 2003. RUP is not a single concrete prescriptive process, but rather an adaptable process framework, intended to be tailored by the development organizations and software project teams that will select the elements of the process that are appropriate for their needs. RUP is a specific implementation of the Unified Process.
- The Scaled Agile Framework (abbreviated as SAFe), is a set of organization and workflow patterns intended to guide enterprises in scaling lean and agile practices. Along with large-scale Scrum (LeSS), disciplined agile delivery (DAD), and Nexus, SAFe is one of a growing number of frameworks that seek to address the problems encountered when scaling beyond a single team.
- Scrum is an agile framework for managing knowledge work, with an emphasis on software development, although it has wide application in other fields and is slowly starting to be explored by traditional project teams more generally. It is designed for teams of three to nine members, who break their work into actions that can be completed within timeboxed iterations, called sprints, no longer than one month and most commonly two weeks, then track progress and re-plan in 15-minute time-boxed stand-up meetings, called daily scrums.
- Scrumban is an Agile management methodology describing hybrids of Scrum and Kanban and was originally designed as a way to transition from Scrum to Kanban. Today, Scrumban is a management framework that emerges when teams employ Scrum as their chosen way of working and use the Kanban Method as a lens through which to view, understand and continuously improve how they work
- SEMAT (Software Engineering Method and Theory) is an initiative to reshape software engineering such that software engineering qualifies as a rigorous discipline. The initiative was launched in December 2009 by Ivar Jacobson, Bertrand Meyer, and Richard Soley with a call for action statement and a vision statement. The initiative was envisioned as a multi-year effort for bridging the gap between the developer community and the academic community and for creating a community giving value to the whole software community.
- In combination with the personal software process (PSP), the team software process (TSP) provides a defined operational process framework that is designed to help teams of managers and engineers organize projects and produce software the principles products that range in size from small projects of several thousand lines of code (KLOC) to very large projects greater than half a million lines of code.
- The Unified Software Development Process or Unified Process is an iterative and incremental software development process framework. The best-known and extensively documented refinement of the Unified Process is the Rational Unified Process (RUP). Other examples are OpenUP and Agile Unified Process.