Home / Technology / AI & IT / DARPA EC developing AI based military sensor fusion technologies to optimally combine information originating from multiple heterogeneous sensors and subsystems.

DARPA EC developing AI based military sensor fusion technologies to optimally combine information originating from multiple heterogeneous sensors and subsystems.

Current artificial intelligence (AI) systems only compute with what they have been programmed or trained for in advance; they have no ability to learn from data input during execution time, and cannot adapt on-line to changes they encounter in real environments.


The AIE program is one key element of DARPA’s broader AI investment strategy that will help ensure the U.S. maintains a technological advantage in this critical area. Past DARPA AI investments facilitated the advancement of “first wave” (rule based) and “second wave” (statistical learning based) AI technologies. DARPA-funded R&D enabled some of the first successes in AI, such as expert systems and search, and more recently has advanced machine learning algorithms and hardware. DARPA is now interested in researching and developing “third wave” AI theory and applications that address the limitations of first and second wave technologies.


DARPA launched AIE program to research about “third wave” AI theory and applications that address the limitations of first and second wave technologies by making it possible for machines to contextually adapt to changing situations.  In the third wave, instead of learning from data, intelligent machines will understand and perceive the world on its own, and learn by understanding the world and reason with it.


The pace of discovery in AI science and technology is accelerating worldwide. AIE will enable DARPA to fund pioneering AI research to discover new areas where R&D programs awarded through this new approach may be able to advance the state of the art. AIE will enable DARPA to go from idea inception to exploration in 90 days.


In May 2023 the Defense Advanced Research Projects Agency (DARPA) issued a Notice of Future Artificial Intelligence Exploration Opportunity: Enabling Confidence. The purpose of this Special Notice (SN) is to provide public notification of additional research areas of interest to DARPA, specifically the Artificial Intelligence Exploration (AIE) program.


The military typically operates in demanding, dynamic, semi-structured and large-scale environments. This reality makes it difficult to detect, track, recognize/classify, and response to all entities within the volume of interest, thus increasing the risk of late (or non-) response to the ones that pose actual threat. A key challenge facing the military operators, in these contexts, is the focus of attention and effort, that is, how to make the most effective use of the available but scarce sensing and processing resources to gather the most relevant information from the environment and fuse it in the most efficient way.


At this time, the DARPA Microsystems Technology Office (MTO) is interested in the following research area to be announced as a potential AIE topic under the Artificial Intelligence Exploration program: Accurate processing of covariance information related to environmental variations and sensor noise is paramount to the performance of statistics-based estimators (e.g., Kalman filters) and is the key enabler for optimally combining information originating from multiple heterogeneous sensors and subsystems.


Objective: EC will develop scalable methods to generate accurate covariance information for the outputs of machine learning (ML) systems to enable enhanced performance when using this information to combine multiple subsystems. The EC AIE will encourage performers to consider a range of ML techniques – e.g., deep learning, Bayesian techniques, etc. – in addressing the following research questions:

1) Can input sensor and environmental covariance information be faithfully reflected in a ML system output covariance matrix in a way that is computationally tractable?

2) Can confident ML subsystems be composed hierarchically to increase inference accuracy, and can they be combined with statistics-based estimation systems (e.g., Kalman filters) to reduce errors?

Challenges: Any approach to achieving the EC objective will need to overcome the following two technical challenges: First, simple covariance measures are often insufficient to characterize output uncertainties after sensor accurately and environmental variations are propagated through the ML architectures. Second, nonlinearities in existing ML architectures make direct calculations intractable, and approximate methods degrade performance.


Structure: EC will be an 18-month effort, with a 9-month Phase 1 and 9-month Phase 2
• Phase 1 will demonstrate the feasibility of generating accurate output covariance baselines for
realistic ML systems by experimentally validating the resulting system covariance model using
cross-correlation measurements.
• Phase 2 will demonstrate the feasibility and utility of edge deployment by: a) demonstrating a
10x speedup in covariance generation (over the Phase 1 techniques); and b) demonstrating
lower errors in an inferencing or estimation task through concatenation of a ML subsystem
with a conventional statistics-based estimator.

Key milestones may include:
• Demonstration that the input sensor model yields a joint probability distribution that is within
5% of the input sensor joint probability distribution, using a suitable performer-proposed
• Demonstration that the baseline system covariance estimates are within 10% of the observed
cross-correlation, using a suitable performer-proposed metric2


Aurora Awarded DARPA Contract to Develop Scalable Methods to Enhance Machine Learning Systems

Aurora Flight Sciences, a Boeing Company, has received a contract award for the Defense Advanced Research Projects Agency’s (DARPA) Enabling Confidence (EC) program. The goal of this program is to develop scalable methods for incorporating uncertainty from sensors and environments into machine learning (ML) system outputs. This uncertainty is key to combining ML approaches with existing statistics-based estimation methods, such as Kalman Filters. As part of this project, Aurora aims to develop and train a novel ML system for simultaneously generating image-based object detections alongside their associated uncertainties.

Under a subaward for this program, Aurora will collaborate with the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) to develop a scalable method to generate and maintain accurate statistical models in state-of-the-art deep neural networks for object detection and tracking. The program aims to enable the Department of Defense (DoD) to exploit advances in AI/ML to develop computationally efficient tools for processing and combining information from multiple sources while providing robust performance in mission-relevant tasks and safety-critical applications.

This is Aurora’s fifth award under DARPA’s Artificial Intelligence Exploration (AIE) program, a key component of the agency’s broader artificial intelligence (AI) investment strategy aimed at ensuring the United States maintains an advantage in this critical and rapidly accelerating technology area.


The agency’s diverse portfolio of fundamental and applied AI research programs is aimed at shaping a future in which AI-enabled machines serve as trusted, collaborative partners in solving problems of importance to national security.


About Rajesh Uppal

Check Also

DARPA ONISQ to exploit quantum computers for improving artificial intelligence (AI), enhancing distributed sensing and improving military Logistics

Quantum technologies offer ultra-secure communications, sensors of unprecedented precision, and computers that are exponentially more …

error: Content is protected !!