Innovation and product development is the cornerstone to organizational growth in the market environment today. But for new product development the stakes are high, requirements increasing and there is a demand of delivering faster in order to beat the competition to the market. New product development is a complex endeavour, which can often be difficult to handle and challenging to see in advance what the end result will be. There is great uncertainty and unexpected
things happen along the way, which affects the scope and direction of a product development project. Therefore these projects can often be difficult to plan, and plans become obsolete soon after their creation.
New ways of managing development emerged within the software industry in the beginning of the 21st century, which revolutionized the way of developing software. Creators of alternative methods in software development came together and formed the Agile Manifesto in 2001. Agile Software Development became a collective term for the new methods, which are all based on iterative and incremental development of self-organizing cross-functional teams, rapidly responding to change These methods have received good reviews from the industry and many large companies use them in their development today. One of the most widely used methods is Scrum, and is currently being used by companies such as Yahoo!, Microsoft, Google and more. Deemer et al. (2010) describe Scrum as “an iterative, incremental framework for projects and product or application development”.
The defense software development process is a highly complex endeavor because it has to deal with a complex domain, it has to provide a very highquality solution and it has to satisfy very specific and complex needs. Command and Control software in particular must meet functional requirements that are often extremely detailed as the situations that create them involve a large number of assets and human resources that have to be coordinated in a way that minimizes losses, as any trade-offs may result in the loss of lives.
Additionally, defense operations are conducted in conformance with a specific set of strategic rules called “doctrine,” and are further specified in very minute detail in a set of operational manuals that are meant to create familiarity in a series of tasks that have to be executed in difficult situations, e.g., under strict time constraint or in the presence of unknown threats. At the same time, considerable freedom must exist to choose the most favorable course of action, given that the situation evolves continuously and often in unpredictable ways. In a command and control system, for example, the leader in charge of the operations must be able to decide the courses of action for any units under his or her control; and these actions must be communicated rapidly and reliably to any and all such units. The feedback loop is also crucial to understand how the situation evolves in the field.
Given the environment in which defense software needs to operate, it is not surprising that until recently, most of the software development process was plan-based: that is, processes that would ensure a high repeatability and predictability. In fact, process standards have been developed by defense agencies and armed forces to prescribe a series of features that every defense software development effort should possess. The old way of developing software, known as the “waterfall” method, involved stages of developing the concept and requirements, analysis, design, construction, testing and deployment, which could take years.
In recent years, however, a number of factors have come to disrupt this plan-based software development process culture inside defense, the most disruptive of which has been the change in the very nature of defense efforts. Whereas traditional conflicts have involved symmetrical opponents, and most doctrines have made that assumption at their very core, recent events prove that more and more conflicts involve asymmetric opponents. Evident examples of this change are the fight against terrorism and organized crime. In these situations, a large and highly structured force is poised against a smaller but nimbler contingent. The two opponents often have profoundly different value systems and cultures, which makes very difficult to construct effective strategies without incurring in uncomfortably high risks.
This rapid change introduced by asymmetrical opponents has a profound impact on the software development effort. Whereas in the past requirements were very detailed and complex but relatively fixed, recent requirements have become more volatile although they remain equally detailed. This dichotomy cannot be addressed using a standard plan-based development process. Another disruptive factor is the cost of software development. While the complexity and volatility of requirements increases, the cost of developing a system also increases accordingly. This increase in cost has become untenable due to a considerable reduction in the defense budget and the need for frequent requirement changes.
In March 2019, the Defense Innovation Board concluded that DoD’s “current approach to software development is broken.” It is a leading source of risk to DoD: it takes too long, is too expensive, and exposes warfighters to unacceptable risk by delaying their access to tools they need to ensure mission success. Instead, software should enable a more effective joint force, strengthen our ability to work with allies, and improve the business processes of the DoD enterprise.”The rise of Cyber warfare and its sophistication, both offensive and defensive, also demands rapid development of effective and secure software at DoD vital for national security.
The DOD is looking to embrace the commercial industry’s trend: open architecture construction, use of data standards, multidomain functionality, applications architecture and an overall architecture that allows for cyber resiliency. Under this method, end user requirements are adopted incrementally and decision-making is more autonomous, resulting in more rapid deployment of scalable, modular software. “Instead of years or months or weeks to develop software or a system, we are now seeing software deployment in weeks or days or hours,” says Probert.
The increasing nature of computing capabilities, the number of technologies that are interconnected to the cyber world, the amount of data generated, and the speed at which data is reported are all reshaping everyday life. To harness this new dynamic, the commercial computer industry has already switched to a more agile way of developing software. More and more, the military is moving to advance the development of cyber-based infrastructure under this changing environment.
Agile emphasizes the ability to react and make adjustments quickly while maintaining a constant pace of development. With agile, innovation becomes an ongoing, sustainable and ultimately rewarding process. Agile involves incremental successes as teams work together and break down projects into manageable sprints. These teams set frequent checkpoints and receive and provide constant feedback, including whether or not capabilities are still valid. The result: projects delivered faster, as expected, and with few surprises upon completion. Agile defense agencies can develop more resilient applications and build cultures centered around innovation and continuous deployment.
The need for agile processes was also reflected in US 2018 National Defense Strategy that called for a more lethal, resilient, and rapidly innovating Joint Force, combined with a robust constellation of allies and partners, that will sustain American influence and ensure favorable balances of power that safeguard the free and national order.
Meanwhile, other countries are taking iterative, natural and agile approaches to large-scale projects. For example, China accelerated the delivery of its J-20 stealth fighter from prototype to deployment in just a few years compared to up to the usual decade. in Italy, this change has been introduced through a new initiative taken at the 4th Logistic Division of the General Staffs of the Italian Army.
Diggerworks, the designer, developer and integrator of combat equipment and clothing used by Australian soldiers, decided to “go Agile.” In the short time since, 100% of Diggerworks teams have shifted from traditional waterfall management to Scrum and other Agile methods. So far: Productivity has improved 400% to 600%, Project delivery times have dropped, in many cases from months to weeks, and employee surveys found that happiness skyrocketed.
Diggerworks was founded in 2011 after Australian Senate Estimates Committee hearings identified critical problems in the procurement and supply chain arrangements of the equipment and clothing used by Australian soldiers.
Diggerworks’ task was to provide better military equipment and clothing, and do it faster. Agile—which uses small, self-governing teams to rapidly solve complex problems—has been essential to meeting that mission. One early Agile success story involves night-vision googles, which can literally be a matter of life or death for combatants. The Australian Army’s standard soft cases for the goggles were not effectively protecting them from wear and tear at the front line. All too often, costly repairs were required and, even worse, soldiers occasionally found themselves with damaged and faulty goggles just when they needed them most. Diggerworks used Agile methods to develop a solution. By adopting the rules of Scrum and with leaders newly trained on Agile management, a hard casing was designed and prototyped within seven weeks. The new casing was $6 million cheaper than the off-the-shelf option and protected googles better, saving an additional $25 million in avoided repairs and replacements.
Given those strong results, it is not surprising other groups are now thinking of adopting Agile. Among them: Army R&D groups, personnel management, and those managing a few major acquisition projects. Even the Royal Australian Navy and Air Force are interested.
Air superiority capability development requires adaptable, affordable and agile processes
Importantly, the rapidly changing operational environment means the Air Force can no longer afford to develop weapon systems on the linear acquisition and development timelines using traditional approaches. Besides being progressively outclassed by fast-improving enemy defenses, America’s fighters have proved increasingly expensive and difficult to develop, buy, and maintain. A single new F-35, currently the Air Force’s only in-production fighters, costs no less than $150 million—tens of millions of dollars more than the older planes it’s replacing.
Air superiority capability development requires adaptable, affordable and agile processes with increasing collaboration between science & technology (S&T), acquisition, requirements and industry professionals. The speed of capability development and fielding will be critical to retain the U.S. advantage in the air.
“There’s no silver bullet,” said Col. Alexus Grynkewich, the Air Superiority 2030 ECCT lead. “We have to match tech cycles –some of them are really long. Engines take a long time to make, but information age tech cycles are fast. Software updates are constantly moving. So how do you move from pacing yourself off industrial age mindsets to information age mindsets?”
The answer, Grynkewich said, is parallel development of maturing technologies for sensors, missions systems, lethality and non-kinetic effects, on appropriate time cycles, of an integrated and networked family of capabilities. The next step is to pull technologies out of each of those parallel efforts when they are ready and developing prototypes, experimenting and gaining more knowledge to determine if the developments are what’s needed in the field.
“What the flight plan lays out is a series of capability development needs, as well as initiatives to prototype and experiment with a number of concepts,” Grynkewich said. “You can start building and then move forward if experimental capabilities are determined to make enough of a difference in highly contested environments of the future
Challenges for agile development in defence
Agile can also be somewhat unpredictable. The federal government likes to know exactly what to expect before embarking on a project: how much it will cost, what it will take to produce, when it will be produced. Waterfall development made these estimates a bit easier, whereas agile is about fluidity.
There is another threat of seeing more and more “Agile in name only” or “fake Agile”. The Department of Defense (DoD) has even issued a guide, “Detecting Agile BS” to help identify and drive out rampant fake Agile. The purpose of the DoD guide is to provide guidance to DoD program executives and acquisition professionals on how to detect software projects that are really using agile development versus those that are simply waterfall or spiral development in agile clothing (‘agile-scrum-fall’).”
The guide offers “flags” for detecting ‘Agile BS’ that include: Software developers not talking to users; Meeting requirements more important than getting something useful to the field quickly; Stakeholders acting “more-or-less autonomously” (e.g. it’s not my job); and Manual processes are tolerated in situations when automation is possible
What is at stake is significant: at a time when cyber warfare, both offensive and defensive, has become at least as important as physical fighting, rapid development of effective software at DoD has become a key ingredient in national security.
Agile to be successful for military must be adopted by organizations effectively. And people must be willing to learn and adjust to the new process, which can be daunting for tenured agency IT personnel. Diggerworks’ Agile experiment could not have worked without a radical shift in leadership style. Though steeped in the military’s rank-and-file mindset, Diggerworks’ leaders were open-minded and invested the time to learn, experiment with and ultimately embrace Agile principles. By asking first what their teams needed from them, they set a new tone and kept their teams focused on a common goal and priorities. By continuing to meet every day to discuss impediments facing their Agile teams, they keep pace with them, and by making their own work and priorities visible, these leaders create greater clarity and accountability to their teams.
They adopted Scrum, the most popular Agile framework. That created a common, neutral methodology and language that helps team members from a variety of traditional silos—Army close combatants, scientists and engineers, among others—work together in a collaborative, rigorous and fast-paced way.
The requirements of agile processes to be successful in military demands that military reduce its hierarchy, streamline bureaucratic processes and bring all the team members collocated either physically or virtually.