DARPA ALIAS fly-by-wire kit turns commercial Helicopters and Aircrafts into Unmanned Drones with autonomous landing

Rajesh Uppal May 17, 2023 AI & IT, Air Force Comments Off on DARPA ALIAS fly-by-wire kit turns commercial Helicopters and Aircrafts into Unmanned Drones with autonomous landing 991 Views

Military aircraft today have evolved over a period of decades to have ever more automated capabilities, improving mission success and safety. At the same time, these aircraft still present challenging and complex interfaces to operators, and despite demanding training regimens, operators can experience extreme workload during emergencies and other unexpected situations. Avionics and software upgrades can help, but can cost tens of millions of dollars per aircraft, which limits the rate of developing, testing and fielding new automation capabilities for those aircraft.

To help overcome these challenges, DARPA launched the Aircrew Labor In-Cockpit Automation System (ALIAS) program. The objective of DARPA’s ALIAS program is to develop and insert new levels of automation into existing military and commercial aircraft to enable those aircraft to operate with reduced onboard crew. The program intends to leverage the considerable advances that have been made in aircraft automation systems over the past 50 years, as well as the advances made in remotely piloted aircraft automation, to help reduce pilot workload, augment mission performance and improve aircraft safety.

The technology aims to improve flight safety and performance and reduce the cognitive load on pilots and the number of onboard crew members with a customizable, drop-in, removable kit that would allow advanced automation to be easily added to existing aircraft.

DARPA successfully tested the effectiveness of ALIAS’ sense and avoid capabilities in 2016 with a Cessna 172G aircraft approaching an unmanned aerial system from multiple angles. Sikorsky has successfully flown a Black Hawk using its autonomous technology for the first time, marking the start of the flight test campaign using the flying testbed helicopter.

The Black Hawk helicopter flew with the full authority fly-by-wire retrofit kit, which removes the mechanical controls from the aircraft, in May 2019 at Sikorsky’s test facility at West Palm Beach, Florida. The successful test flight is one of the first steps towards the unmanned autonomous flight of helicopters. Follow-on testing of the optionally-piloted aircraft aims to expand the flight envelope of the aircraft until the fully autonomous flight is achieved, said the company. If successful, the initiative would allow the military to make better use of pilots’ time. It would also allow the military to get more use out of the aircraft if it could go on low-risk missions while pilots rested.

“Testing mission scenarios allows transition partners to see those benefits firsthand,” according to Root. “The final DARPA demo plans to demonstrate uninhabited flight or fully autonomous flight from taxi to takeoff to landing and shutdown. This will require a robust capability not only to avoid obstacles, but also to respond to in-flight emergency procedures if needed. This maps well to future mission needs in support of single pilot operations.”

Chris Van Buiten, vice president of Sikorsky Innovations said, “This technology brings a new dimension of safety, reliability and capability to existing and future helicopters and to those who depend on them to complete their missions.

DARPA funded ALIAS at nearly $17.2 million in fiscal 2018 and $10 million in fiscal 2019 but has requested no funding for the program in fiscal 2020. The latter “decrease reflects program completion,” according to the Pentagon justification book on the fiscal 2020 DARPA budget request.

“A fundamental tenant of the ALIAS framework is portability across platforms,” DARPA said. “The program has already successfully demonstrated this portability across fixed and rotary wing aircraft, both military and commercial. We anticipate we will have additional information available later this year regarding potential integration for military aircraft.”

This follow-on research would shrink the system size; further test the ranging and collision-avoidance features; mature additional capabilities of the system such as detecting aircraft below the horizon and in poor light conditions; and improve calculations for optimal aircraft trajectories to avert impending collision.

The system could ultimately serve as a line of defense in future layered air-traffic management systems that could include Automatic Dependent Surveillance-Broadcast (ADS-B) transponders and ground-based radar systems that are part of the federal NextGen effort. There is particular potential applicability for unmanned air systems or aircraft with reduced crew sizes.

In future, a pilot will not only fly his own aircraft, but will also control many others and, while managing that formation, artificial intelligence (AI) will help fly the pilot’s own aircraft.

These formations could be fixed wing or rotorcraft, engaging in air defense suppression or anti-armor attacks – the human pilot will be kept (relatively) safe by sending in the autonomous platforms first. This is part of a concept of operations vision for the United States Army’s Future Vertical Lift (FVL) program and the US Air Force’s Mosaic Warfare concept.

It’s a concept that DARPA describes as being like ceramic tiles in mosaics; individual warfighting platforms put together to make a larger picture – or, in the Air Force’s case, a hugely complex attack formation. The idea is to send so many weapon and sensor platforms, manned and unmanned, that enemy forces are overwhelmed; turning complexity into an advantage.

DARPA has been working to find a way to realize this ‘Mosaic Warfare’ and gives the example of two formations of aircraft, one a four-ship of piloted aircraft and the other inexpensive, expendable drones; their aerial mix overcoming the enemy.

ALIAS Seeks to Provide Portable, Flexible Advanced Autopilot Capabilities

As an automation system, ALIAS aims to support execution of an entire mission from takeoff to landing, even in the face of contingency events such as aircraft system failures. ALIAS’ goals include producing a customizable, drop-in, removable kit so fewer onboard crew members will be needed on military aircraft.

The integrated sense-and-avoid (SAA) system includes a single optical camera that provides imagery for detection and tracking. The system also incorporates passive ranging features that assess the likelihood of an incoming aircraft intersecting the flight path of its host aircraft, and collision-avoidance capabilities to determine the best way to steer the host aircraft out of harm’s way.

Through ALIAS, pilots can fly the aircraft using a tablet computer that recognizes familiar prompts such as swiping and tapping. Sikorsky’s demonstration system, which connects to existing mechanical, electrical and diagnostic systems, fits under the cabin floor and within the airframe of both airplanes and helicopters.

ALIAS system attributes, such as persistent-state monitoring and rapid recall of flight procedures, would further enhance flight safety. Easy-to-use touch and voice interfaces would facilitate supervisor-ALIAS interaction. ALIAS would also provide a platform for integrating additional automation or autonomy capabilities tailored for specific missions.

“Our goal is to design and develop a full-time automated assistant that could be rapidly adapted to help operate diverse aircraft through an easy-to-use operator interface,” said Daniel Patt, Darpa program manager, adding this would transform the pilot from system operator to mission director. The portable kit would control sufficient aircraft functionality to permit the automation to manage the complete flight, including handling contingency events such as systems failures, allowing the operator to act as a mission commander, Darpa said.

As an automation system, ALIAS would execute a planned mission from takeoff to landing, even in the face of contingency events such as aircraft system failures. “Pilots can choose to engage autonomy to help operate, dynamically plan, adjust and/or execute a complete spectrum of responsibilities, allowing the operators to better focus on the designated mission at hand,” said Mark Miller, Sikorsky vice president of engineering and technology. “Our autonomy capabilities will help pilots in high-workload and degraded-visual environments, ultimately increasing safety and efficiency.”

A single-pilot test has been the most challenging to pull off, according to Lt. Col. Philip Root, acting deputy director of the Tactical Technology Office. There’s no feedback loop for the pilot because co-pilot is a relatively invisible, silent machine. “You now have a co-pilot that’s not there, and [pilots] don’t know how to rely on someone who’s not there,” he said. “How does a machine have the same contextual understanding of when to talk, so to speak, and what information is relevant? And how does the human pilot know when to trust the autonomy?”

But despite humans’ natural distrust for machines, ALIAS aims to bridge that gap by having pilots train with the technology, becoming familiar with it like they would with a new iPhone. “Pilots, operators, Marines trust those things that work,” Root said. “Trust is two parts: You have to believe the system can actually deliver, and you have to see it deliver routinely. Those two things are separate,” he said. “We can provide ability to trust a machine if we develop it from the ground up to foster that trust.” Whether that trust is earned remains to be seen.

ALIAS targets advancement in three key technical thrust areas:

(1) Minimally invasive interfaces from ALIAS to existing aircraft: It is anticipated that the ALIAS system would need to operate aircraft functions to provide automated operation. Systems generally confined to the cockpit would support the vision of portability.

(2) Knowledge acquisition on aircraft operations: To support adaptation of the ALIAS toolkit across different aircraft in a short amount of time, it is anticipated the ALIAS system would benefit from the leverage of existing host aircraft procedural information, existing flight mechanics information or models, or other methods of rapidly developing requisite aircraft information.

(3) Human-machine interfaces: A vision for ALIAS is that the human operator provides high-level input consistent with replanning and mission-level supervision and is not engaged in lower-level flight maintenance tasks that demand constant vigilance.

Highlights of ALIAS’ flexible architecture include:

The potential for integration onto multiple fixed and rotary-wing platforms, both military and commercial;
Cockpit displays and human interfaces that support reduced workload and/or reduced crew, as well as improved safety, such as terrain avoidance;
Full coverage of typical aircrew tasks and emergency procedures;
The ability to integrate directly with existing air vehicle systems, subsystems, and mission payloads;
Redundancy and software assurance to support certification for human occupancy; and
The ability to rapidly integrate new applications including third-party algorithms and applications onto existing aircraft.

https://www.youtube.com/watch?v=RDzsHMEAzF0

ALIAS PHASE-1

Sikorsky, a Lockheed Martin Company, successfully demonstrated a 30-mile autonomous flight using a Sikorsky S-76 commercial helicopter to complete Phase 1 of an $8 million award from the Defense Advanced Research Projects Agency (DARPA)’s Aircrew Labor In-Cockpit Automation System (ALIAS) program.

This flight highlighted the ability for an operator to plan and execute every phase of an autonomous mission with a tablet device. During the demonstration, a ground station crew located at the flight initiation field monitored the progress of the ALIAS-enabled Sikorsky Autonomy Research Aircraft (SARA), an S-76 commercial helicopter.

“With the advances we’ve made, the capability for safe, unobtrusive optionally piloted flight is here,” said Mark Miller, Vice President of Engineering & Technology at Sikorsky. “ALIAS is expanding the role of optionally piloted helicopters for early entry into established aircraft programs. It has the capability of not only reducing aircrew size, but also changing the type and length of training required for safe operation.”

“The current environment limits the creation of new, optionally piloted platforms. What Sikorsky and DARPA are demonstrating is the successful and affordable integration of advanced technology onto existing legacy aircraft to not only set the stage for autonomous operations down the road, but also to immediately improve aircraft performance, reduce maintenance costs, and increase crew and passenger safety,” said Chris Van Buiten, vice president of Sikorsky Innovations, the technology research group for the engineering and technology organization. “We are delighted to be working with DARPA on this transformational program,” said Van Buiten.

ALIAS’ Phase 2 accomplishments

DARPA awarded Sikorsky a $9.8 million modification for the competition’s second phase, which focuses on continued maturation of the initial ALIAS system with additional flight tests, enhancements to the human interface and transition to additional aircraft to demonstrate ALIAS portability.

Successful flight demonstrations of ALIAS technology installed in two different Cessna 208 Caravan fixed-wing aircraft, a Diamond DA-42 fixed-wing aircraft, and a Sikorsky S-76 helicopter
Successful ground demonstrations of ALIAS responding to various simulated flight contingency events, such as system failures, that might cause pilots to deviate from pre-set plans or standard courses of action
Demonstration of quickly tailoring ALIAS to new platforms, and showing that installation and removal of the kit did not impact airworthiness

ALIAS makes use of in-cockpit machine vision, robotic components to actuate the flight controls, an advanced tablet-based user interface, speech recognition and synthesis, and a knowledge acquisition process. The latter facilitates transition of the automation system to another aircraft within a 30-day period.

The company points out in its list of system features: “Ability to learn aircraft procedures and visually gather information without requiring access to aircraft avionics.” “In Phase 2, we exceeded our original program objectives with two performers, Sikorsky and Aurora Flight Sciences, each of which conducted flight tests on two different aircraft,” said Scott Wierzbanowski, DARPA program manager. “In Phase 3, we plan to further enhance ALIAS’ ability to respond to contingencies, decrease pilot workload, and adapt to different missions and aircraft types.

We’re particularly interested in exploring intuitive human-machine interface approaches—including using handheld devices—that would allow users to interact with and control the ALIAS system more easily. Ultimately, we want to design for and demonstrate the improved ALIAS system across as many as seven previously untested fixed- and rotary-wing platforms.”

Jessica Duda, Humans and Autonomy Group Lead at Aurora Flight Sciences, was quoted in Digital Trends. “ALIAS as a whole is much more capable than an autopilot, as it includes procedures tracking and monitoring, contingency identification and response, and intelligent interaction with the onboard pilot.” The National Aeronautics and Space Administration (NASA), the U.S. Air Force, the U.S. Army, and the U.S. Navy have all expressed interest in ALIAS’ potential capabilities and are providing support to the program.

Stress is a major factor in combat; it can alter how people act. As voice commands and natural language are expected to be used with ALIAS, as well as touch screen input, stress becomes an important issue. “We have to be able to deal with that, you have to deal with other stressful things like bad language or unexpected out-of-domain language,” Young said. ‘Out of domain’ is a reference to the limited vocabulary used to interact with the ALIAS AI. “We tend to call that structured language. They have to use a specific set [of words] to make the language translation better,” Stuart Young, head of the Aircrew Labor In-Cockpit Automation System (ALIAS) program explained.

In Young’s view, ALIAS could be applied to the F-16 or another fourth-generation type, and also be used to “augment your fifth-generation fighter”. He sees other directions in the application of ALIAS technology, including reducing the number of pilots in the aircraft. During Phase II of ALIAS, experiments were conducted with a robot arm that operated the controls of an aircraft, acting as a co-pilot to a human pilot. Those trials were carried out with US aerospace technology developer Aurora Flight Sciences in 2016 and 2017

ALIAS Phase III

Now in Phase 3, the Sikorsky engineers developing ALIAS have begun to integrate the system into a UH-60 Black Hawk for testing and flight demonstration in 2019. As the biggest fleet of aircraft in the Army and widely relied on by the Department of Defense, Drozeski said the Black Hawk is the ideal platform for ALIAS to quickly benefit service partners.

“We’ve chosen the Black Hawk as the platform we want to demonstrate full integration of ALIAS-type capabilities – all the circuit breakers and switches and instruments in the aircraft, so that the capability ALIAS provides to a crew member is really like a co-pilot,” said Drozeski. “It can fly routes, plan routes, execute emergency procedures, and do all that perfectly.”

The U.S. Defense Advanced Research Projects Agency (DARPA) awarded Sikorsky a contract to carry out a third phase of its program to develop an Aircrew Labor In-Cockpit Automation System (ALIAS), the Lockheed Martin subsidiary announced in January. Under Phase III of the program, Sikorsky will focus on the fabrication and installation of prototype kits on multiple aircraft variants, and conduct flight demonstrations for each.

The drop-in, kit-based system provides advanced automation to an existing aircraft; the aim of the ALIAS program is to reduce pilot workload while enhancing mission performance and safety. In addition to assisting with decision making on a manned aircraft, DARPA and industry officials see the technology being applied to control unmanned aircraft systems.

Through the DARPA ALIAS program, Sikorsky is developing systems intelligence that will give operators the confidence to fly aircraft safely, reliably and affordably in optimally piloted modes enabling flight with two, one or zero crew. The program will improve operator decision aiding for manned operations while also enabling unmanned operations.

ALIAS Demonstrations

DARPA in 2016 proved the effectiveness of the effort’s sensory and avoidance capabilities with a Cessna 172G aircraft approaching an unmanned aerial system from multiple angles. “This is all a part of operationalizing autonomy,” Lt. Col. Philip Root, acting deputy director of the Tactical Technology Office, said. “Those words really mean something to me.”

US Army pilots have demonstrated the use of what they are calling “supervised autonomy” to direct an optionally-piloted helicopter through a series of missions to demonstrate technology developed by Sikorsky, a Lockheed Martin company and the Defense Advanced Research Projects Agency (DARPA).

“The Army refers to this as Mission Adaptive Autonomy. It’s there when the pilot needs the aircraft to fly itself and keep it free of obstacles, so the pilot can focus on more of the mission commander type role. But the pilot is able to interact with the system to re-suggest, re-route or re-plan on the fly,” said Lt. Col. Carl Ott, chief of Flight Test for the U.S. Army Aviation and Missile Research, Development and Engineering Center’s Aviation Development Directorate.

During the hour-long flight demonstration, Ott interfaced with the autonomous capabilities of the system to conduct a series of realistic missions, including aircrew tasks such as low-level terrain flight, confined area takeoffs and landings, landing zone selection, trajectory planning, and wire-obstacle avoidance.

An S-76B commercial helicopter flew over a small crowd gathered at Fort Eustis, Virginia, landed in an adjacent field after adjusting to miss a vehicle, and rose up to hover perfectly motionless for several minutes. The mid-October 2018 demonstration was remarkable because the pilot carried out the maneuvers using supervised autonomy in an aircraft equipped with DARPA’s Aircrew Labor In-Cockpit Automation System (ALIAS). He operated the system via novel control interceptors and a tablet he had used for the first time just three days beforehand.

“Hovering in adverse winds is a task that consumes a human pilot’s attention, but automated flight control achieves ‘rock steady’ precision,” said Graham Drozeski, the DARPA program manager for ALIAS, explaining how offloading pilots’ cognitive burden frees them to focus on mission execution. “Really, we want the pilot’s eyes and mind on the fight rather than holding an altitude. That’s the core focus of ALIAS: bringing the latest advances from unmanned aircraft into a piloted aircraft through an interface that provides fluid interaction with the autonomous capabilities.”

SARA, which has more than 300 hours of autonomous flight, successfully demonstrated the advanced capabilities developed as part of the third phase of DARPA’s Aircrew Labor In-Cockpit Automation System (ALIAS) program. The aircraft was operated at different times by pilots on board and pilots on the ground. Sikorsky’s MATRIX Technology autonomous software and hardware, which is installed on SARA, executed various scenarios including:

Automated Take Off and Landing: The helicopter autonomously executed take-off, traveled to its destination, and autonomously landed.
Obstacle Avoidance: The helicopter’s LIDAR and cameras enabled it to detect and avoid unknown objects such as wires, towers and moving vehicles.
Automatic Landing Zone Selection: The helicopter’s LIDAR sensors determined a safe landing zone.
Contour Flight: The helicopter flew low to the ground and behind trees.

Sikorsky has been flight testing its S-70 Optionally Piloted Vehicle (OPV) Black Hawk in West Palm Beach, Fl., and the company and L3Harris are resolving data link issues disclosed after the first flight of the OPV in May 2019. The S-70 OPV is a UH-60A retrofitted with Matrix’s full-authority, fly-by-wire flight controls.

Mark Ward, Sikorsky’s chief pilot, said this month that Sikorsky and L3Harris have fixed many of the data link issues related to flight test telemetry data, since the first flight test but that “we’re still not up to the maximum range we’d like.”

Sikorsky has said that OPV technology has been through more than 300 hours of flight testing through DARPA’s ALIAS program on rotorcraft and fixed wing platforms. Igor Cherepinsky, Sikorsky’s director of autonomy, has said that Sikorsky’s goal in fielding OPV technology is preventing Controlled Flight Into Terrain and Degraded Visual Environment issues and accidents.

DARPA, Sikorsky plan to expand autonomous capability to navy rotorcraft

Programme officials at the Defense Advanced Research Projects Agency (DARPA), along with their counterparts at Lockheed Martin-owned Sikorsky, are planning to expand a DARPA-led programme for autonomous US Army rotorcraft into US Navy-operated aircraft.

“We are actively talking with the navy, along with our [other] customers” about transitioning the artificial intelligence (AI) and autonomous flight technology from ALIAS into other rotorcraft platforms, specifically the CH-53K, Igor Cherepinsky, director of Sikorsky Innovations said during a November 2022 briefing on the DARPA programme. The DARPA and Lockheed Martin/Sikorsky team conducted three separate autonomous flights in October with the ALIAS system aboard the Black Hawk.