Unmanned Aerial Vehicles (UAVs) are growing at frentic pace driven by civil, consumer and military requirements. According to Volpe report, the number of UASs operating in the U.S. National Airspace System (NAS) will exceed 250,000 by the year 2035. There are growing number of civil and commercial applications of UAVs, including humanitarian aid and disaster relief, infrastructure monitoring (such as oil pipelines), wildlife conservation and precision agriculture.
Tech titans like Uber, Amazon, and Google have all laid out ambitious plans for filling the skies with autonomous aircraft. Uber plans to launch an “on demand aviation” service called Uber Elevate through its flying car project. The European aerospace giant Airbus recently unveiled its secret flying-car project dubbed Vahana — a single-manned, autonomously piloted aircraft that can take off and land vertically. Amazon and Google plans to launch automated drone delivery fleets across urban areas that could eliminate the need for shipping via post or UPS. One of the critical technology to these plans is reliable sense and avoid technology.
Safe and efficient integration of Unmanned Aircraft Systems (UAS) into Civil Airspace is a key challenge for unleashing their potential for non-military applications “without reducing existing capacity, decreasing safety, impacting current operators, or placing other airspace users or persons and property on the ground at increased risk”
UAS Traffic Management (UTM) technologies and the associated regulatory framework to allow unrestricted access of UAS to all classes of airspace, including very low-level and Beyond-Line-of-Sight (BLoS) operations. Recent developments in communications, navigation and Sense-and-Avoid (SAA) technology are progressively supporting UAS operations in medium-to-high density operational environments, including urban environments.
UAVs have also become indispensable to modern militaries in providing intelligence, near-real time reconnaissance and surveillance to commanders, and offering warfighters greater battlespace awareness. They have proven effective in electronic combat support, battle damage assessment and even in national security operations like border surveillance, low intensity conflict and guerilla / terrorist warfare. In the future, UAVs will act as airborne data links, enemy radar jammers, chemical and biological weapons detectors, target acquisition systems, and finally precision air attack systems.
In order for a UAS to safely navigate in the already crowded aerial environment of the modern world, the U.S. Federal Aviation Administration (FAA) and other international organizations have mandated that unmanned aircraft must have an on-board Sense and Avoid (SAA). SAA can be defined as the capability of a UAS to remain well clear from and avoid collisions with other airborne traffic. SAA provides the intended functions of self-separation and collision avoidance as a means of compliance with the regulatory requirements to “see and avoid” compatible with expected behaviour of aircraft operating in the airspace system.
Till recently, UAVs could not autonomously detect or avoid other UAVs, aircraft or obstacles such as buildings, and therefore presented a severe concern for mid-air collisions. As a consequence, they were not allowed to be flown out of line of sight or within close proximity to large gatherings of people, thus restricting their uses within the commercial sector.
However, the Sense and Avoid technology, also referred to as ‘detect and avoid’, sense and avoid’ or ‘collision avoidance’ technology has now become matured and Drones with obstacle detection and collision avoidance sensors are becoming more prevalent in both the consumer and professional sectors. Sense and Avoid may utilize Stereo Vision, Monocular Vision, Ultrasonic, Infrared, Time-of-Flight and Lidar sensors to detect and avoid obstacles. Manufacturers are also using multiple sensors fusing them together to create the obstacle detection and collision avoidance systems.
This obstacle detection and avoidance technology started with sensors detecting objects in front of the drone. Now the latest drones from DJI, Walkera, Yuneec and others have front, back, below and side obstacle avoidance sensors.
Sense and Avoid technology for autonomous UAVs and Swarms
Among the several safety threats in airspace operation, mid-air collision can be highlighted, which depends on a set of events despite issues in aircraft mechanical systems, such as high ATCo workload levels and loss of the minimum separation established between aircraft. For example, bugs in software may maneuver the aircraft and assign it to undesired headings. Also, considering RPAS, failures in the Command and Control (C2) link, i.e., the link the pilot uses to communicate to the aircraft, may lead to unsafe states
Sense and avoid technology enables Drones to detect aircraft & obstacles within the vicinity of the UAV and to execute manoeuvres to restore a safe situation if needed. In addition UASs must ensure that they can avoid the terrain and land without operator intervention, react to contingencies such as engine out and lost link scenarios, and Be reliable and cost-effective. Sense and avoid is a sequence of functions which, using a combination of airborne and ground-based sensors, are able to perform manoeuvres to avoid collisions and serve as a UAV replacement for the tradition “see and avoid” capability for manned aircraft.
The Sense and avoid technology is of two types, one is for drones called manual sense and avoid which relays information to the UAV pilot and the second is the development of completely autonomous sense and avoid technology which removes the need for a pilot altogether. The cooperative technologies depend on cooperation from other aircrafts to know their distance, velocity and altitude and avoid collisions, while noncooperative technologies use active and passive sensors to determine these parameters on their own.
The future autonomous systems will have ability to perform with a higher level of autonomy. Instead of just being able to execute a set of pre-programmed functions, they will be better able to react to their environment and perform more situational-dependent tasks as well as synchronized and integrated functions with other autonomous systems.
UAV Swarms is another emerging technology that could prove revolutionary for Military Strategies. These swarms can find, fix, and communicate precise target location of ground, sea, and air targets; they can serve as weapons platforms to attack air defense systems from multiple axes; or they can pass missile targeting data to any platform carrying a counter air missile. Sense and avoid challenges are even more complicated for autonomous systems and swarms.
Obstacle Detection And Collision Avoidance Technology
The latest high tech drones are now equipped with collision avoidance systems. These use obstacle detection sensors to scan the surroundings, while software algorithms and SLAM technology produce the images into 3D maps allowing the drone to sense and avoid. These systems fuse one or more of the following sensors to sense and avoid; Vision Sensor, Ultrasonic, Infrared, Lidar, Time of Flight (ToF) and Monocular Vision. The latest DJI Mavic 2 Pro and Mavic 2 Zoom have obstacle sensing on all 6 sides. The Mavic 2 uses both Vision and Infrared sensors fused into a vision system known as Omni-directional Obstacle Sensing.
The DJI Mavic 2 obstacle sensing system is top drone technology. The Mavic 2 will sense objects, then fly around obstacles in front. It can do the same when flying backwards. Or hover if it is not possible to fly around the obstacle. This technology is known as APAS (Advanced Pilot Assistance System) on the DJI Mavic 2 and Mavic Air drones.
In December 2019, the Skydio 2 drone was released. This also has obstacle avoidance on all sides. The Skydio 2 autonomy technology visualizes and calculates what’s happening around the drone. It can then intelligently predict what will happen next and will make accurate decisions multiple times a second. The Skydio 2 quadcopter uses 6 x 4k cameras to build a 3D map of its surroundings, which will include trees, people, animals, cars, buildings and more.
NASA DAA system for Unmanned Aircraft
NASA has signed contracts with three industry partners in a bid to demonstrate progress in the use and eventual certification of systems critical to the safe operation of Unmanned Aircraft Systems (UAS) in the National Airspace System (NAS). The demonstration is known as the Systems Integration and Operationalization (SIO) activity and will take the form of a number of flight tests during the summer of 2020 using various-sized unmanned aircraft built by each of the three companies.
“These are not the smaller drones much of the public is used to hearing about. These are larger UAS that will fly above 500 feet to simulate missions such as pipeline inspections or cargo delivery to off-shore oil platforms,” said Robert Sakahara, NASA’s project manager for the UAS Integration in the NAS Project. The companies now under contract include Bell Helicopter Textron, Inc., of Fort Worth, Texas; General Atomics Aeronautical Systems, Inc., of Poway, California; and PAE ISR, LLC of Sterling, Virginia.
NASA and these industry partners will work together to tackle key challenges that prevent routine commercial UAS operations today, including development, integration, and certification of unmanned aircraft and its onboard systems. “We’re using our expertise within these areas to help our partners move toward certification,” Sakahara said.
The two key systems that are the focus of the demonstration that will hopefully assist the Federal Aviation Administration toward setting standards for certification include Detect and Avoid (DAA) and Command and Control (C2). DAA involves employing sensors of some type (such as radar or cameras) to sense if the aircraft is flying too close to an object (such as a tall building or another aircraft) and then takes steps to fly away from the potential danger. C2 involves technology that ensures the unmanned aircraft remains in constant, secure contact via radio with ground-based pilots and air traffic control – but also knows what to do on its own to stay safe in case that signal is lost.
NASA has developed technology that may enable unmanned aircraft to fly safely in the national airspace along with piloted aircraft through its program called Unmanned Aircraft Systems in the National Air Space or UAS in the NAS. The patent-pending integrated communications and control system is capable of collision warnings as well as real-time traffic and weather updates.
NASA recently tested on remotely piloted Ikhana aircraft its prototype Detect-and-Avoid (DAA) system working in concert with airborne and ground-based computers. Ikhana made 11 flights involving more than 200 scripted encounters with approaching aircraft. Depending on the specific scenario, either Ikhana detected one or more approaching aircraft and sent an alert to its remote pilot to take action, or Ikhana itself took action on its own by flying a programmed maneuver to avoid a collision – an aviation first.
General Atomics Aeronautical Systems, Inc. has developed one of the three primary DAA sensors flown on Ikhana, in this case a prototype radar system. It also contributed Ikhana system and self-separation and collision avoidance alerting logic software. The other two sensors included an Automatic Dependent Surveillance – Broadcast (ADS-B) from BAE Systems, and a second generation Traffic alert and Collision Avoidance System (TCAS) from Honeywell International, Inc.
Vigilant Aerospace Systems intends to commercialize the technology as part of its new FlightHorizon product suite and equip manned and unmanned aircraft with the hardware and software that provides synthetic cockpit views and detect-and-avoid commands to improve flight safety for all kinds of aircraft.
“One of major advantages of this system is that it uses existing FAA infrastructure to help keep drones away from other aircraft,” said Kraettli L. Epperson, CEO of Vigilant Aerospace Systems. “It also gives nearby aircraft the ability to be aware of the drone and improves situational awareness for the drone operator.”
DARPA’s ALIAS program
DARPA launched Aircrew Labor In-Cockpit Automation System (ALIAS) program with objective to develop and insert new levels of automation into existing military and commercial aircraft to enable those aircraft to operate with reduced onboard crew. The technology aims to improve flight safety and performance and reduce the cognitive load on pilots and the number of onboard crew members with a customizable, drop-in, removable kit that would allow advanced automation to be easily added to existing aircraft.
DARPA’s Aircrew Labor In-Cockpit Automation System (ALIAS) program has conducted the first successful flight tests of a shoebox-sized, plug-and-play system designed to enable manned and unmanned aircraft to automatically detect nearby aircraft and avoid potential mid-air collisions. An unmanned air vehicle (UAV) repeatedly used the technology demonstration system to detect and track in real time a Cessna 172G aircraft approaching from various vertical and horizontal distances.
The integrated sense-and-avoid (SAA) system includes a single optical camera that provides imagery for detection and tracking. The system also incorporates passive ranging features that assess the likelihood of an incoming aircraft intersecting the flight path of its host aircraft, and collision-avoidance capabilities to determine the best way to steer the host aircraft out of harm’s way.
The work is part of a DARPA effort to create a low-cost, easily installed system to detect oncoming or crossing aircraft and determine the best avoidance strategy compliant with standard rules that set minimum vertical and lateral distances between aircraft during flight.
This follow-on research would shrink the system size; further test the ranging and collision-avoidance features; mature additional capabilities of the system such as detecting aircraft below the horizon and in poor light conditions; and improve calculations for optimal aircraft trajectories to avert impending collision.
The system could ultimately serve as a line of defense in future layered air-traffic management systems that could include Automatic Dependent Surveillance-Broadcast (ADS-B) transponders and ground-based radar systems that are part of the federal NextGen effort. There is particular potential applicability for unmanned air systems or aircraft with reduced crew sizes.
RDRTec developing common Sense and Avoid for Navy’s Fire Scout and Trinitron
RDRTec has announced that the U.S. Navy MQ-4C Triton Unmanned Aircraft System (UAS) program has approved proceeding with Phase 3 of the Small Business Innovation Research (SBIR) program to advance the company’s Common Radar Airborne Collision Avoidance System (C-RACAS) for Triton. Naval Air Warfare Center Aircraft Division-Lakehurst in Lakehurst, N.J., awarded a $3 million contract to RDRTec for developing common Radar Autonomous Collisions Avoidance System (RACAS) Sense and Avoid (SAA) technology that fulfills both the Fire Scout and Triton unmanned aerial vehicles (UAVs).
The Fire Scout is an unmanned autonomous helicopter designed to provide reconnaissance, situational awareness, aerial fire support, and precision targeting support for ground, air, and sea forces. The Triton is a maritime version of the Northrop Grumman RQ-4C Global Hawk long-range high-altitude reconnaissance UAV adapted for maritime patrol in a support role to the Navy P-8 Poseidon manned maritime patrol jet.
In a separate Navy project, RDRTec experts developed radar sense-and-avoid technologies for the Fire Scout UAV using advanced AESA technology and proprietary signal processing to provide actionable collision warning information with lead times longer than 30 seconds involving non-cooperative aircraft moving as fast as 400 knots.
C-RACAS is a C-Band Radar system that provides the UAS with Sense and Avoid (SAA) and Weather Avoidance capabilities. C-RACAS leverages 10 years of US Navy investment to provide the MQ-4C UAS with effective situational awareness of non-cooperative aircraft to enable collision avoidance (CA). This can be integrated with the future Airborne Collision Avoidance System For Unmanned Aircraft (ACAS Xu) system. The system determines airborne object location and estimated flight path, so that timely action to maintain safety may be taken by the air vehicle operator. C-RACAS features long range search, detection and track functionality over a wide Field of Regard (FoR). The C-RACAS system is designed to minimize size, weight, power and cooling at an affordable cost
RDRTec Inc. is developing adaptive multi-channel phased array manifold radar technology that reconfigures by radar mode for optimum performance. Company experts are focusing on multi-mode X or C-band radars with phased arrays that support modes with high and low bandwidth requirements. High bandwidth modes include synthetic aperture radar (SAR), inverse synthetic aperture radar (ISAR), and high range resolution (HRR). Relatively low bandwidth modes include ground moving target indicator (GMTI), maritime moving target indicator (MMTI), air-to-air (AA) and sense and avoid (SAA) modes
USAF desires Remotely Piloted Aircraft (RPA) Sense and Avoid (SAA)
The Air Force has released a request for information notice requesting sense and avoid technologies for large RPAs – Predators and Global Hawks – though the notice said that sense and avoid technologies for smaller RPA such as the hand-launched Wasp and Raven are also encouraged.
As weapon systems have become increasingly complex, so have the environments in which they are expected to operate. Remotely Piloted Aircraft (RPA) Sense and Avoid (SAA) capabilities enable RPAs to maintain safe separation to include avoiding collisions as well as safely integrate with other airspace users across the full range of operations in global airspace environments. The global airspace includes operations in National Airspace (NAS), International Civil Aviation Organization (ICAO)/international airspace to include Due Regard, and Military Airspace (MAS).
Federal Aviation Rule (FAR) 91-113 requires the pilot to “see and avoid” other traffic. Because RPAs have no onboard pilot, RPA require a SAA capability which enables the RPA to operate safely in airspace with other users. The SAA capability must be able to provide safe-separation to include avoiding collisions.
The solution responses are expected to include, but not limited to, descriptions for the following attributes:
- Strategies for airworthiness certification (e.g. MIL-HNBK-516 C, or most current) and operational approval of the proposed solution
- Information Assurance and Cyber Security implementation
- Solution characteristics of Open Systems / Open Architectures which are highly desirable for a SAA solution
- Strategies for adherence / certification to quality management practices (e.g. SAE AS9100C, “Quality Management Systems: Requirements for Aviation”)
- Strategies for reliability and maintainability
- Strategies for system security and survivability
- Strategies for addressing Critical Program Information (CPI)
While working sense and avoid technology still appears part of the distant future, the main consensus emerging from those researching the technology, including the ASTRAEA and MIDCAS Projects, is that the basic technology is there, it just needs to be monitored and developed.