International Defense Security & Technology Your trusted Source for News, Research and Analysis
Related Articles
Military decision-making applications could include strategy development, course of action planning, tactical analyses, intelligence gathering, wargaming and simulation, deception detection, coalition formation and evolution, and others. Such domains typically involve extremely large search spaces (making even a single traversal of a complete decision tree impossible), private information that is never revealed, sequential revelation of information about an opponent’s resources and intentions, use of deception to mask resources and intent, continuous resource quantities that can be committed at every decision point, stochastic outcome distributions from a given set of resources and actions, and the ability to learn and improve from one iteration to another as each opponent develops a more extensive and sophisticated set of observations about the others’ choices in past iterations.
DARPA is interested in exploring the value to military decision making of recent developments in artificial intelligence and computational game theory applied to ongoing series of competitive and adversarial interactions between homogeneous and/or heterogeneous agents with imperfect information in massive search spaces. These developments have potential applicability to many different domains such as business competition, medical treatment planning, cybersecurity, and others, and have been demonstrated and evaluated in games, particularly poker (computer versus computer and also computer versus human).
The goal of Serial Interactions in Imperfect Information Games Applied to Complex Military Decision-Making (SI3-CMD) is to demonstrate the feasibility of applying techniques from artificial intelligence, game theory, and decision sciences to complex military decision making. In particular, Phase 1 is intended to produce a detailed feasibility analysis and Phase 2 an initial pilot demonstration on a real or realistic problem that includes most if not all of the characteristics described in the Introduction.
DARPA envisions that current AI and game theoretic techniques will initially be evaluated, augmented, and applied to competition between two agents; however, we are also interested in extensions to situations where more than two agents are interacting. These multi-agent situations may involve homogeneous or heterogeneous agents, may allow agents to observe others’ actions even when the observer is not part of an interaction, may present opportunities for alliances and other forms of cooperative behavior, and may admit emergent effects where the intentional or unintentional behaviors of some agents may have unintended effects on the outcomes of other agents. Military conflicts involving multiple parties with distinct motivations and shifting alliances are an example of these types of situations.
Serial Interactions in Imperfect Information Games Applied to Complex Military Decision Making (SI3-CMD) builds on recent developments in artificial intelligence and game theory to enable more effective decisions in adversarial domains. SI3-CMD will explore several military decision-making applications at strategic, tactical, and operational levels and develop AI/game theory techniques appropriate for their problem characteristics.
These applications will extend current AI/game theory techniques to be effective when there are multiple interacting agents, extremely large search spaces, sequential revelation of information, use of deception, continuous resource quantities, stochastic outcomes, and the ability to learn from past iterations. The program will produce new techniques and assessments of their effectiveness for military uses
DARPA awards Boeing company to develop Human-Level, Decision-Making Artificial Intelligence to Help Strategize, Engage, and Adapt in Multi-Agent Environments
The Boeing Company has been awarded a new project to develop an artificial intelligence (AI) decision-making engine for multi-agent military problems such as multi-domain wargaming and strategic battle management. The goal of the project — known as the Adversarial Game-based Iterative Learning Engine (AGILE) — is to learn effective tactics and behaviors by playing adversarial games, such as 2v2 air-to-air engagement, at a human performance level. It is hoped that AGILE will enable the development of advanced reinforcement learning (RL) algorithms, a type of machine learning that does not require any human-labeled data to train an AI agent.
The AGILE goal is a fully autonomous system that will make high-level decisions that humans usually make: “Recent advances in RL have used these types of algorithms to train AI agents to play against humans and win board games, such as Go or chess, that have very well-defined moves to specific places on a board,” said senior researcher Deepak Khosla, HRL’s principal investigator on AGILE. “Our AI strategies are similar to the ones developed for those games, but we are aiming to expand to multi-agent games with imperfect information and numerous variations that far exceed those in a deterministic game with perfect information such as Go or chess.”
The key innovation the HRL team hopes to introduce is reinforcement learning that exhibits robustness, scalability, and adaptability; in the meantime, The Boeing Company will focus on developing a variety of multi-agent scenarios and simulations plus validation of AGILE’s performance. The AGILE goal is a fully autonomous system that will make high-level decisions that humans usually make. “We’re ambitious with this system,” Khosla said. “Our goal is to show that a team of AI agents can play games to learn autonomous behaviors and human-like strategies in adversarial military engagement problems.”
The key innovation the HRL team hopes to introduce is reinforcement learning that exhibits robustness, scalability, and adaptability. The Boeing Company will focus on developing a variety of multi-agent scenarios and simulations as well as validation of AGILE performance. The AGILE project is part of Serial Interactions in Imperfect Information Games Applied to Complex Military Decision Making (SI3-CMD), a program of the Defense Advanced Research Project Agency, or DARPA, focused on applying recent developments in artificial intelligence and game theory to complex military decision making in changing multi-agent environments.
