International Defense Security & Technology Your trusted Source for News, Research and Analysis
Related Articles
Ai/ATR is a generic term to describe automated processing functions carried out on imaging sensor data in order to perform operations ranging from simple cuing of a human observer to complex, fully autonomous object acquisition and identification.
ATR can range from fully autonomous, such as, in a missile seeker to aided target recognition (AiTR) processing that presents image annotations to the human observer to make the final decision as to the importance and veracity of the information generated and the action to be taken.
Advancements in EOIR sensor technology have enabled integration of cameras with higher resolutions, improved sensitivity, and multi-spectral imaging onto ground vehicle platforms. These sensors play a critical role in movement, situational awareness and target acquisition in combat environments day and night and have become an integral part of the warfighter’s capabilities.
Current imaging sensors however largely rely on soldier’s continued attention on the image/video display. An abundance of sensors and the complexity of tasks in complex environments has made this a daunting task for the Soldier.
Recent advances in image exploitation, artificial intelligence and machine learning coupled with a surge in demand for increased autonomy of ground platforms have reinvigorated the interest in automatic target detection, recognition, identification and tracking technologies.
US Army plans to use AiTD and AiTR on manned platforms to help reduce Soldier workload, improve situational awareness, and reduce response times. On unmanned ground platforms, AiTD and AiTR become a fundamental enabling technology for autonomous operation and mission execution.
Traditional AiTD and AiTR algorithm development has focused on Moving Target Indication (MTI) and Static Target Indication (STI) of military targets in relatively unpopulated low clutter rural environments.
While this is still an important function, in the future, manned and unmanned ground vehicle platforms will be operated in increasingly complex environments, to include high clutter rural environments such as vehicles or ATGM teams in defilade; as well as urban areas where the enemies may blend in with natural patterns of life. Future AiTD and AiTR approaches will be required to perform basic detection and recognition functionalities in this environment, but will also need to provide more advanced automated behaviors to discriminate and prioritize potential threats.
Significant technological challenges exist to successfully utilizing traditional computer vision and emerging machine learning technologies in operational combat environments. Some of these challenges include robustness and reliability of algorithms, SWAP-C of computational hardware, and limited data communication and computational bandwidth. However, arguably a bigger challenge to successfully deliver of AiTR/AiTD capabilities to the warfighter has been the mismatch between the operational requirement/expectation of the soldier and the capabilities and readiness of the AiTR/AiTD technology, says US Army
Military Ai / ATR technology
All three services are engaged in research and development for reliable Ai/ATR capabilities for myriad combat missions. Army, Navy, and Air Force are pursuing Ai/ATR with sensor packages for their respective platforms to do the following: reconnaissance, intelligence, surveillance, target acquisition, fire control, wide-area search and track, countermine, and sensor fusion. Change detection and MTI that relates to target disposition are also of interest.
Army sensor assets typically emphasize EO/IR because of sensor size, weight, and power constraints on the platform, whereas Navy and Air Force tend to emphasize high range resolution and SAR radars due to the long stand-off ranges associated with ship and aircraft engagement ranges.
An Ai/ATR operates on sensor data in order to process information for decision making. The primary value added to a weapons system of an Ai/ATR is engagement timeline reduction for target(s) acquisition. The rapid and reliable acquisition and servicing of targets increase lethality and survivability of the weapons platform/soldier.
There are many military scenarios where a reliable Ai/ATR capability would provide an enormous capability to the soldier. Persistent surveillance (PS) relies on things that change in a scene and use change-detection algorithms and moving-target indication (MTI). Change detection can be a major tool in improvised explosive detection (IED) detection. Disturbed earth, where a device has been buried, presents a significantly different signature than undisturbed earth. The disturbed earth presents a much more uniform, blackbody like, spectral signature compared to the much more structured signature of undisturbed soil. Extremely large coverage areas, such as that required in PS or for airborne detection of IEDs along a roadway, with sufficient resolution and update rate become driving sensor parameters.
Ai/ATR can also enable the overcoming of unmanned air-/ground-vehicle bandwidth limitations by selection for transmission of only target information to a weapons platform. A reliable onboard Ai/ATR would select and send only target information back to the unmanned air vehicle (UAV) operator without the enormous data bandwidth for transmission of the complete scene over the flight path from which the operator must extract the target.
Munitions precision targeting and lock-on-after-launch seekers are other examples of fully autonomous ATR. The need for ground-to-ground Ai/ATR in urban environments is amplified due to the huge fields of regard (∼2π steradians), the shortness of timelines, and the need to discriminate combatant from noncombatant. The Ai/ATR task difficulty is extremely task dependent, and a canonical data set is always a concern for training and evaluation in a military scenario.
There is a whole hierarchy of possible tasks that can be of interest for an Ai/ATR algorithm. The level of discrimination can cover a whole gamut, from detection to classification to recognition to identification.
Today, one of the most difficult tasks of interest is identification of intent. Whereas, in the past, detection of a human may have been sufficient, today the soldier must also determine the intent of the human detected. Is the intent of the detected human hostile?
US Army Military requirements
U.S. Army posted a request for information for Aided Target Detection and Recognition Technologies for Manned and Unmanned Ground Vehicles in Complex Environments. The primary objective of this RFI is to canvas a wide community of traditional and non-traditional providers of technology solutions and services to help identify current state of art in the areas of aided and automatic target detection, recognition and tracking for Electro Optical Infra-Red (EOIR) sensors.
This includes, but is not limited to, machine learning and deep learning approaches for real-time exploitation and enhanced situational awareness for ground-based manned and unmanned platforms operating in complex rural and urban environments.
The primary emphasis of the RFI is to identify Aided Target Detection (AiTD) and Aided Target Recognition (AiTR) algorithms, image and video processing, machine vision, and sensor exploitation technologies for manned and unmanned ground vehicles to enable increasing levels of artificial intelligence. However, new and emerging computing hardware technologies aimed at SWAP-C constrained implementation of the state of the art sensor exploitation algorithms in real-time are also of interest.
AiTR/AiTD for ground vehicles is ultimately desired to work on sensors for both static and on-the-move vehicles in real-time in diverse environments. The desired product must be able to be fully integrated with EOIR sensors on manned and unmanned military ground vehicles (High Mobility Multipurpose Wheeled Vehicle (HMMWV), Bradley, Abrams, Stryker, MRAP, NGCV, etc.) while observing their specific space constraints and operational environments.
The following is a short list indicative of the possible capabilities that may be of interest for both narrow field of view (FOV) targeting sensors as well as wide FOV search and/or situational awareness sensors.
– Automated search and detection for military targets of interest. This includes, but is not limited to, vehicles, personnel, weapons systems, and UAS that may be fully exposed or partially obscured.
– Automated recognition or classification of targets to include military versus civilian vehicles, weapon/no weapon discrimination, and facial recognition.
– Automated multi-target tracking.
– Algorithms that provide enhanced discrimination, tracking, and other advanced capabilities in dense urban settings.
– Algorithms compatible with high definition uncooled IR sensors used for 360˚ situational awareness.
– Algorithms compatible with existing fielded Second Generation (2GF) FLIR sensors.
– Algorithms that leverage multi-spectral sensors, such as Third Generation (3GEN) FLIR.
– Collaborative AiTD/AiTR, between ground vehicle sensors or ground vehicle sensors and small UAS.
– EOIR data sets including ground-to-ground militarily relevant targets, as well as solutions to collect and/or generate relevant imagery with groundtruth or labels for training
Sensor Oriented Requirements
Next generation IR cameras have large formats such as 1280 × 960, 1920 × 1200, and 2K × 2K with typical frame rates at 30 Hz to 60 Hz. EO cameras are available in even higher pixel formats and frame rates. Multi-spectral sensors such as 3GEN FLIR are also becoming available.
While the objective is to ultimately develop technology that can work day/night on infrared sensors, responses that have focused on visible sensor development are acceptable to show development status. Responses that exploit multi-spectral sensors are encouraged as well. AiTR and AiTD capabilities are desired on both narrow FOV targeting sensors as well as wide FOV search and/or situational awareness sensors.
The intent of this RFI is to identify an ecosystem of algorithms, exploitation capabilities, and machine vision approaches that can be leveraged (within reason) with a wide variety of sensor types including but not limited to EO and IR sensors of different spatial, temporal, spectral, and radiometric resolutions.
Computing Hardware Oriented Requirements
While the primary emphasis of this RFI is on algorithms, software implementation and architectures to enable real-time AiTR/AiTD capabilities, it is also important to understand the ecosystem of the computing hardware on which these capabilities may be hosted.
In that spirit, submission on new and evolving computing hardware architectures for efficient, low Size, Weight, Power, and Cost (SWAP-C) computing is encouraged. Submission on specific embedded, FPGA or ASIC implementation of well-defined visual processing and exploitation capabilities or tools that can be embedded in a larger system level solution are also encouraged.
Although the primary purpose of this hardware is to execute AiTD/AiTR software, this hardware may interact with related on- and off-platform systems. As such, technologies that support low-latency distribution of real-time sensor video from high bandwidth EO/IR sensors to multiple consumers (human or machine) are of interest. Approaches that enable rapid real-time prototyping, adaptation, and implementation of algorithm approaches are also of interest.
Operational and Environmental Requirements
AiTR/AiTD algorithms will be required to be effective in a broad array of environmental conditions on ground vehicle platforms of various sizes and configurations. The following list highlights some operational and environmental considerations for algorithm performance:
– Broad array of rural and urban environments with varying degrees of natural and man-made clutter
– Static or moving targets, partially obscured targets
– Targets over a broad range of operationally relevant ranges, i.e., close to the vehicle and at significant standoff
– Real-time operation with both static and on-the-move platforms
-Tracking persistence for slewable sensors
