To Agile, or not to Agile, that is the Question class=

To Agile, or not to Agile, that is the Question

Posted on in Government Digital Transformation

Outrageous fortune of waterfall processes in government

“Whether ’tis nobler in the mind to suffer
The slings and arrows of outrageous fortune,
Or to take arms against a sea of troubles,
And by opposing end them?” – Hamlet by William Shakespeare
Many government IT initiatives operate like Greek tragedies: every effort to reduce project risk ends up increasing risk. And, high failure rates, especially when adapting Enterprise Resource Planning (ERP) software, originally developed for the private sector, in government.
There are also Shakespearean tragedy overtones in government IT. Governments and donor partners often impose project management “best practices” on providers like FreeBalance. One of these “best practices” is the imposition of waterfall-type project management that pre-supposes that software projects are as predictable as construction projects. Construction benefits from physical constraints like the strength of materials. Software has far fewer constraints, and higher customization opportunity.
FreeBalance Waterfall Project Design
Imposed waterfall practices increases “outrageous fortune”. Can agile project management increase success ratios?

Waterfall battalions of sorrows

“When sorrows come, they come not single spies. But in battalions!”  – Hamlet by William Shakespeare
A recent survey by Accenture and NASCIO (National Association of State Chief Information Officers) found  that “when asked what outcomes could be avoided by using agile, 70 percent of IT professionals felt it helped avoid wasted money from ineffective IT projects, 66 percent felt it helped avoid large IT project failures and 58 percent said it helped prevent programs that do not meet business needs.”
Agile techniques such as lean, design thinking, Extreme, Kanban, and Scrum have often been associated with small projects. The thinking, among traditional observers, is that agile is not appropriate for large IT projects. As a Mitre study for the United States Department of Defense pointed out, waterfall projects incorrectly assume well-defined requirements with limited change. The reality is that agile is much more resilient to change and uncertainty than waterfall. Agile is ideal for substantial Government Resource Planning (GRP) initiatives. That’s because of insights gained during project implementation that can better align functionality with real government needs. Most GRP system implementation follow government reform and modernization. It’s not just IT, it’s transformation.
This is where agile excels.
Agile scales. From small to large projects.

To agile, or not to agile? An ABCs checklist

“There are more things in heaven and earth, Horatio,
Than are dreamt of in your philosophy.”  – Hamlet by William Shakespeare
Not all waterfall projects fail. Not all agile projects succeed. (To be fair, agile focuses on learning, so failing fast is often a success factor.)
The FreeBalance Strategy and Innovation Group has analyzed project literature and our government experiences. (Unlike legacy enterprise software vendors, FreeBalance participates in all implementations to help drive product and service improvements.) This analysis has led to our A-i3+qM methodology. It also led to a significant revaluation of traditional practices, and the philosophy of these practices.
We’ve also created a 26 point checklist rating of factors. Factors are rated between low and high. Your context could be different.
Factors Favouring Waterfall A-F
Agile or Waterfall in GRP Checklist
Factors Favouring Agile G-Z
Factors Favouring Agile G-Z

  1. Architectural Complexity: need to develop software architectures from scratch tend to require long design periods beyond multiple iterations – although this doesn’t apply to commercially-available architectures like the FreeBalance Accountability Platform
  2. Problems Well-Understood: deep insight into problems reduces the need to gather additional insight from agile – although, many (most?) large government IT projects are solution-focused that rarely address real problems without changes
  3. Experience in Similar Circumstances: government organization experience implementing similar initiatives makes projects more predictable – although, many government organizations assume vendor experience out of context such as vendor experience in other contexts
  4. Number of Personnel: agile operates well with small motivated teams who operate efficiently – although it should be noted that waterfall projects require more personnel for communications, coordination and documentation than agile
  5. Scope Focus: fully articulated project scope, and a focus on scope driving time and resources, lends itself to controls in waterfall that reduces “scope creep” – although, many projects fail because of unbending scope changes favouring contract provisions over real government needs
  6. Human Capacity: high government human capacity in technology, project management, and Public Financial Management (PFM) mitigates many of the risks associated with waterfall – although high capacity is often associated with overconfidence
  7. Project Complexity: agile helps decompose complexity and structure activities based on priorities through prototypes, user input, and design analysis
  8. Project Uncertainty: agile is ideal when there is high project uncertainty
  9. IT Failure Rates: agile provides success tools for IT teams who have experienced a history of underwhelming results
  10. Cross-Domain Functions: waterfall works best in silos by engaging domain specialists, and agile functions best when functions cross silos such as finance, procurement, payroll, and tax administration
  11. Outcome-Focused: projects conceived to have results benefit from agile processes because waterfall projects focus on contractual compliance
  12. Transformational Change: projects that follow modernization, reform, or re-organization benefit from agile processes that are more transparent, communicative, and flexible than waterfall
  13. Custom Development: expectation for custom development (such as custom development using a technical platform, custom development using a governance platform, or custom development using ERP) benefits from agile that identifies value for every custom element
  14. Current Legacy Technology Level: government organization that use ancient technology (mainframes, COBOL, 4GLs), or legacy technology (most proprietary ERP systems, including Tier 1), benefit from the fresh eyes associated with agile
  15. User Involvement: agile excels when users are required to articulate problems, identify solutions, test functionality, and championing new systems
  16. Functionality Change: agile excels when there are many new functions will be introduced compared to legacy system
  17. Greenfield Implementation: upgrades to existing systems can be implemented in waterfall, while net-new software implementation benefit from agile
  18. Requirements Change: waterfall expects very little change to requirements, while agile is resilient to change by identifying specification changes early to reduce costs compared to identifying needed changes later in projects
  19. System Dependencies: waterfall processes benefit from fewer system dependencies, such as integration points with other systems, while agile is better able to integrate across multiple government domains, and underlying technologies
  20. Product Novelty: agile is ideal when projects are implementing relatively new product suites
  21. Change Resistance: waterfall is not effective in situations where user and managerial resistance to change is high, while agile has more built-in processes that facilitate and integrate change management (bolt-on change management in waterfall projects is often ineffective)
  22. Quality Focus: waterfall processes put testing and quality assurance late in the methodology, while agile focuses on quality in each “user story” resulting in improving quality early
  23. System Extensibility: agile excels when existing or new software needs to extend to additional functions through reuse
  24. Custom Reports & Dashboards: the articulation of reports including replicating statutory reports in new software, and creating new reports, dashboards, and analysis benefits from agile iterations because users often recognize improvements once seeing report outputs
  25. Implementation Speed: agile excels in implementation speed, and provides more effective performance information (“cadence”) than waterfall, to predict project completion time
  26. Project Transparency: waterfall assumes separate teams who report around milestones, while agile assumes constant project transparency through Kanban, Scrum or Srumban boards, and frequent engagement with users and stakeholders

Important Notes

  • The diagrams above show that agile is preferable in the first 2 columns of the checklist. That’s not a mistake.
  • Your checklist is likely to show items in all 3 columns. Use this as a risk

Puzzling vendor-led “agile” methodologies

“It puzzles the will.” – Hamlet by William Shakespeare
Many established enterprise software vendors, including Tier 1 ERP vendors, are touting agile processes. Systems integrators need to support these vendor-driven processes in order to achieve certification. It’s startling how unagile these so-called agile processes are.
Agile is often bolted onto waterfall by adding process complexity. Or, describing milestone communication presentations as agile. Many of these processes impose “best practices” in functions that are rarely best, or appropriate. Project acceleration in these methodologies focus on running what’s in the software without change, rather than matching to real government needs. (“Vanilla” implementations are rarely satisfying to government needs.)

An agile rose?

“A rose by any other name would smell as sweet” – Romeo and Juliet by William Shakespeare
Many providers try to redefine waterfall as agile. This won’t change the nature of the process. It won’t smell as sweet as agile.

Addditional Sources

Government-Specific References

General References

Topics

Contact