DARPA SC2 challenge use AI to optimize spectrum usage in Wireless Networks and Adaptive Radios to cooperatively share or dominate congested spectrum

Rajesh Uppal November 28, 2019 AI & IT, Electronics & EW Comments Off on DARPA SC2 challenge use AI to optimize spectrum usage in Wireless Networks and Adaptive Radios to cooperatively share or dominate congested spectrum 1,088 Views

Ongoing wireless revolution is fueling a voracious demand for access to the radio frequency (RF) spectrum around the world. In the civilian sector, consumer devices from smartphones to wearable fitness recorders to smart kitchen appliances are competing for bandwidth. Around 50 billion wireless devices are projected to be vying for access to mobile communications networks within the next few years and by 2030, the demand for wireless access could be 250 times what it is today. However, as the use of wireless technology proliferates, radios and communication devices often interfere with and disrupt other wireless devices.

Military spectrum requirements are also increasing exponentially as military operations increasingly rely on access to the wireless spectrum in order to assess the tactical environment and coordinate and execute their critical missions. The demand for more and timely information at every echelon is driving an increase in DoD’s need for spectrum.“Increasingly lower echelons, including individual soldiers, require situational awareness information resulting in more spectrum-enabled network links.” Managing this increasing demand, while combating what appears to be a looming scarcity of RF spectrum is a serious problem for our nation, both militarily and economically, says DARPA.

However, spectrum is a finite resource and additionally DOD has to free up 500 MHz of the spectrum it has for commercial use by 2020 leading to scarcity of spectrum for DOD use. Therefore Spectrum congestion is becoming a growing problem, DARPA officials explain. It increasingly limits operational capabilities due to the increasing deployment and bandwidth of wireless communications, the use of network-centric and unmanned systems, and the need for increased flexibility in radar and communications spectrum to improve performance and overcome sophisticated countermeasures.

Currently the spectrum is managed by nearly a century old technique, by isolating wireless systems by dividing the spectrum into exclusively licensed bands, which are allocated over large, geographically defined regions. This approach rations access to the spectrum in exchange for the guarantee of interference-free communication. However, allocation is human-driven and not adaptive to the dynamics of supply and demand. At any given time, many allocated bands are unused by licensees while other bands are overwhelmed, thus squandering the spectrum’s enormous capacity and unnecessarily creating conditions of scarcity.

The current situation also poses potential security risks for the military, creating the impression of reliable and unfettered access to the spectrum while in actuality creating a well-defined target for adversaries that may wish to disrupt wireless operations. First responder radios need to be able to communicate reliably in such congested and contested environments and to share radio spectrum without direct coordination or spectrum preplanning.

In March 2016, DARPA launched the Spectrum Collaboration Challenge (SC2), an initiative designed to ensure that the exponentially growing number of military and civilian wireless devices will have full access to the increasingly crowded electromagnetic spectrum. These networks will be capable of intelligently optimizing the spectrum by collaborating with, and learning from, the other systems that occupy the spectrum with them. SC2 competitors will reimagine spectrum access strategies and develop a new wireless paradigm in which radio networks, will autonomously collaborate and reason about how to share the RF spectrum, avoid interference, and jointly exploit opportunities to achieve the most efficient use of the available spectrum.

DARPA announced in Oct 2019 that GatorWings, a team of undergraduate students, Ph.D. candidates, and professors from the University of Florida are the winners of the Spectrum Collaboration Challenge (SC2) – a three-year competition to unlock the true potential of the radio frequency (RF) spectrum with artificial intelligence (AI). GatorWings’ autonomous radio was able to navigate the various wireless obstacles developed for SC2 to thoroughly stress each team’s AI-enabled radios. GatorWings’ unique approach to the SC2 challenge helped it eke out the competition. Using an AI engine that is one-step beyond basic rule-based systems, GatorWings applied foundational reinforcement learning AI techniques to optimize each “pocket” of available spectrum.

While GatorWings took home the top spot, the second and third place finishers were MarmotE and Zylinium, respectively. MarmotE, a team of current and former Vanderbilt researchers, took home the $1 million second place prize, while the third place prize of $750,000 went to Zylinium, a three-person start-up with expertise in software-defined radios (SDRs) and AI. Andersons, a two-person team of hobbyists and SDR enthusiasts that also successfully competed in DARPA’s 2014 Spectrum Challenge, and Erebus, a three-person company created specifically to tackle SC2, rounded out the top five.

“It was truly a battle right until the end, with GatorWings beating out MarmotE by just one point. Each team took a slightly different approach to the final scenarios – some used AI to navigate the wireless spectrum like a driverless car, while others used machine learning to promote competing or collaborating solutions. In the end, the three highest ranked teams were able to maximize their use of the spectrum by skillfully collaborating with their competitors’ radios while successfully completing as many data transfers as possible,” said Tilghman.

Spectrum Collaboration Challenge administrator Paul Tilghman said: “SC2 sets out to bring the software defined radio and artificial intelligence communities together to fundamentally rethink 100 years of spectrum practice, and tackle the original and enduring spectrum grand challenge: efficient coexistence of all wireless communications.

DARPA’s grand challenges are actually a really good exploratory measure, when you have a very tough problem, but one that’s very tangible, and also one where there’s many possible different types of solutions to that problem. And what you’re really searching for is a way to sift through a large number of potential solutions and neck it down to the one or two or three that are really viable. And again here we find ourselves in the Spectrum Collaboration Challenge with a large number of possible solutions for how you bring intelligence into the radio, and we’re really hoping that by using a competition, that we’re able to determine really which strategies, what kinds of artificial intelligence, are best at optimizing the spectrum under every circumstance.

Using AI for Spectrum management

Before SC2, various DARPA projects had demonstrated that a handful of radios could autonomously manage spectrum by frequency hopping, as Bluetooth does, in order to avoid one another. The first Bluetooth devices struggled to avoid interfering with Wi-Fi routers, a higher-powered, more-established cohort on the radio spectrum, with which Bluetooth devices shared frequencies. Bluetooth engineers eventually modified their standard—and saved their wireless tech from early extinction—by developing frequency-hopping techniques for Bluetooth devices, which shifted operation to unoccupied bands upon detecting Wi-Fi signals.

Unfortunately, frequency hopping works only up to a point. It depends on the availability of unused spectrum, and if there are too many radios trying to send signals, there won’t be much, if any, unused spectrum available, says Paul Tilghman program manager.

The DARPA Spectrum Challenge aims to stimulate the development of innovative approaches to adaptive, software-based radio communications in such multi-user environments. Teams will compete to create protocols for software-defined radios that best use communication channels in the presence of other dynamic users and interfering signals. The Challenge is not focused on developing new radio hardware, but instead seeks algorithmic strategies for guaranteeing successful communication in the presence of other radios without explicit coordination.

To make SC2 work, we realized, we would need to test competing teams on scenarios with dozens of radios trying to share a spectrum band simultaneously. That way, we could ensure that each radio couldn’t have its own dedicated channel, because there wouldn’t be enough spectrum to go around.

With that in mind, we developed scenarios that would be played out in a series of round-robin matches, in which three, four, or five independent radio networks all broadcast together in a roughly one-square-kilometer area. The radio networks would be permitted access to the same frequencies, and each network would use an AI system to figure out how to share those frequencies with the other networks. We would determine how successful a given match was based on how many tasks, such as phone calls and video streams, were completed. A group of radio networks completing more tasks than another group would be crowned the winner for that match. However, our main goal was to see teams develop AI-managed radio networks that would be capable of completing more tasks collectively than would be possible if each radio was using an exclusive frequency band.

We realized quickly that placing these radios in the real world would have been impractical. We would never be able to guarantee that the wireless conditions would be the same for each team that competed. Also, moving individual radios around to set up each scenario and each match would have been far too complicated and time consuming.

So we built Colosseum, the world’s largest radio-frequency emulation test-bed. Currently housed in Laurel, Md., at the Johns Hopkins University Applied Physics Laboratory, Colosseum occupies 21 server racks, consumes 65 kilowatts, and requires roughly the same amount of cooling as 10 large homes. The Challenge runs in Colosseum, a simulated environment hosted on a supercomputing cluster. DARPA constructed the largest-of-its-kind wireless testbed, which serves during and after the SC2 as a national asset for evaluating spectrum-sharing strategies, tactics, and algorithms for next-generation radio systems.

For each match, we plug in radios so that they can “broadcast” radio-frequency signals straight into Colosseum. This test-bed has enough computing power to calculate how those signals will behave, according to a detailed mathematical model of a given environment. For example, within Colosseum are emulated walls, off which signals “bounce.” There are emulated rainstorms and ponds, within which signals are partly “absorbed.”

The emulation provides all the information necessary for the teams’ AIs to make appropriate decisions based on their observations during each emulated scenario. Faced with a cellphone jammer that is flooding a frequency with meaningless noise, for example, an AI might choose to change its frequency to one not affected by the jammer.

It’s one thing to build an environment for AIs to collaboratively manage spectrum, but it’s another thing entirely to create those AIs. To understand how the teams competing in SC2 are building these AI systems, you need a bit of background on how AI has developed in the past several decades.

Broadly speaking, researchers have advanced AI in a couple of “waves” that have redefined how these systems learn. The first wave of AI was expert systems. These AIs are created by interviewing experts in a particular area and deriving a set of rules from them that an autonomous system can use to make decisions while trying to accomplish something. These AIs excel at problems, such as chess, where the rules can be written down in a straightforward fashion. In fact, one of the best-known examples of first-wave AI is IBM’s Deep Blue, which first beat chess master Garry Kasparov in 1997.

There’s a newer, second wave of AI that relies on huge amounts of data, rather than human expertise, to learn the rules of a given task. Second-wave AI is particularly good at problems where humans have trouble writing down all the nuances of a problem and where there often seem to be more exceptions than rules. Recognizing speech is an example of such a problem. These systems ingest complex raw data, such as audio signals, and then make decisions about the data, such as what words were spoken. This wave of AI is the type we find in the speech recognition used by digital assistants like Siri and Alexa.

Today, neither first- nor second-wave AI is used for managing wireless spectrum. That meant that we could consider both waves of AI and the ways in which researchers teach those AIs how to solve problems, to find the best solution to our problem. Ultimately, it is easiest to treat spectrum management as a reinforcement-learning problem, in which we reward the AI when it succeeds and penalize it when it fails. For example, the AI may receive one point for successfully transmitting data, or lose one point for a transmission that was dropped. By accumulating points during a training period, the AI remembers successes and tries to repeat them, while also moving away from unsuccessful tactics.

In our competition, a dropped transmission often happens because of interference from another radio’s transmission. So we also have to think of wireless management as a collaborative challenge, because there are multiple radios broadcasting at the same time. The key to AI-managed radios performing better than traditional, static allocation is developing AIs that can maximize their own points while leaving room for the other AIs to do the same. Teams are rewarded when they make as many successful transmissions as possible without constantly bumping into one another in the pursuit of available spectrum, which would prevent them all from maximizing use of that spectrum.

This new breed of collaborative intelligent radio networks could give rise to a rich spectral ecosystem able to accommodate an enormous diversity of communicating devices while operating 100 to 1,000 times more efficiently than today’s wireless networks

DARPA Spectrum Challenge SC2

Defense Advanced Research Projects Agency (DARPA) Spectrum Collaboration Challenge (SC2) will use a series of tournament events to spur development of next-generation wireless networks which make more effective use of the RF spectrum.

A Vanderbilt team of researchers and alumni – dubbed MarmotE – won the Round 1 in mid-December of the U.S. Defense Advanced Research Projects Agency’s Spectrum Collaboration Challenge (SC2), leading the top 10 teams, each awarded $750,000 in prize money. This was the first event of the three-year long tournament. Round 2 is set for December 2018.

“Right now, we’re really in the middle of the competition,” Tilghman continued. “We’re in our second phase, we have 20 active competitors. In our first phase we had 30 active competitors. On December 12, DARPA held the second preliminary event of the Spectrum Collaboration Challenge (SC2). At the event’s conclusion, six of the eight top-scoring teams walked away with $750,000 each in prize money.

During the PE2 matches, teams were put through six different RF scenarios designed to mimic the challenges that collaborative, autonomous radios will face in the real world. These scenarios challenged the radios to collaboratively mitigate interfering with an incumbent radio system, sense and adapt to the spectrum demands of high-traffic environments, handle the data demands of the connected soldier of the future, and beyond. Each scenario was designed to pressure test various elements of the teams’ approaches and, in particular, their ability to successfully collaborate with the other radios operating within the same environment. “During the second preliminary event we witnessed a technological shift,” said Paul Tilghman, the DARPA program manager leading SC2. “For the first time, we saw autonomous collaboration outperform the status quo for spectrum management.”

The competition now enters its third year and moves closer to the finale, which will be held at one of the country’s largest annual technology and telecommunications shows – MWC19 Los Angeles. And in that live stage show, we’re going to be competing the teams head to head, and at the end of the stage show, we’ll be awarding $3.75 million in prizes, $2,000,000 prize to the top placing team.

The competition will unfold in three year-long phases beginning in 2017 and finishing, for those teams that survive the two Preliminary Events, in a high-profile Championship Event in late 2019. All 30 teams will have to meet several requirements throughout the year to prepare for the Preliminary Event #1 Competition in December 2017. Top performers during phase 1 will proceed to phase 2 next year, which culminates in another event competition in December 2018. The DARPA has selected 30 teams for Phase I of the Spectrum Collaboration Challenge.

The third and final phase, to be held until the final competition at the end of 2019, will award $2 million, $1 million, and $750,000 prizes, respectively, to the top three finishers. The team whose advanced, software-defined radios collaborate most effectively with a diversity of simultaneously operating radios in a manner that optimizes spectrum usage for the entire communicating ensemble will walk away with a grand prize of $2 million.

New DARPA Grand Challenge to Focus on Spectrum Collaboration

The win is especially significant for Peter Volgyesi, a research scientist and Miklos Maroti, a research associate professor at Vanderbilt’s Institute for Software Integrated Systems. For the preliminary event, 475 fully autonomous matches were run with the 19 qualified teams’ radio designs in SC2’s custom testbed environment, known as Colosseum. The final matches for the first event were carried out across six different communications scenarios designed to mirror real-world congested environments, but with more complexity than existing commercial radios are equipped to handle.

The competing teams faced fluctuating bandwidths and interference from other competitors as well as DARPA designed bots that tested and challenged their radio designs. Each team’s radio performance was scored based on its collaborative spectrum sharing abilities.

“Central management of the spectrum is simply not scalable and pretty wasteful, but ad-hoc sharing as implemented in WiFi is not working either,” said Maroti. “The best solution to spectrum management would be a combination of distributed cooperation and adaptation driven by the latest advances of machine learning.”

The Round 1 competition found that when two radio networks were asked to share the spectrum, the top performing teams were successful at adapting their spectrum usage so that both networks could successfully transmit with minimal interference.

Fully autonomous sharing of the spectrum with three simultaneous wireless technologies however, remains a difficult challenge. When three different technologies attempt to coexist simultaneously there is a smaller set of overlapping strategies that will fulfill each individual radio network’s needs. This causes conflict and requires a higher degree of agility and reasoning, which will be required to be successful in the next phase.

SC2 teams will take advantage of recent advances in artificial intelligence and machine learning and the expanding capacities of software-defined radios to develop breakthrough capabilities that can help bring about spectrum abundance.

After two intense days of competition, teams from Tennessee Technological University and Georgia Tech Research Institute and an independent team of individuals emerged as the overall winners, earning a total of $150,000 in prize money. The agency’s Spectrum Collaboration Challenge (SC2) will reward teams for developing smart systems that collaboratively, rather than competitively, adapt in real time to today’s fast-changing, congested spectrum environment—redefining the conventional spectrum management roles of humans and machines in order to maximize the flow of radio frequency (RF) signals.

The challenge is expected to both take advantage of recent significant progress in the fields of artificial intelligence and machine learning and also spur new developments in those research domains, with potential applications in other fields where collaborative decision-making is critical.

Both the preliminary and final events included two separate tournaments, each with its own goals:

Cooperative tournament:

In each match, three teams attempted to effectively share the spectrum while transmitting random data files from their source radio to their destination radio over the same 5 MHz UHF band. A team’s match score was its total packets delivered plus the higher of the two other teams’ delivered packets—thus motivating cooperative behavior.

Teams could not coordinate in advance on how to share the spectrum; instead, they had to develop and implement algorithms to enable their assigned software-defined radios to dynamically communicate at a high rate while leaving spectrum available for the other two teams to do the same. This event tested conditions encountered during military operations involving multiple units and coalition partners, and also has possible future commercial applications.

Competitive tournament:

In each match, two teams sought to dominate the spectrum, with the winner being the first to transmit all its files of random data (or to successfully transmit the most packets in three minutes) from a source radio to a destination radio. Teams had to develop and implement algorithms to enable their assigned software-defined radio to dynamically communicate at a high rate in the presence of competitors’ signals within the same 5 MHz UHF band. This event tested conditions directly applicable to military communications, where radios must deliver high-priority data in congested and often contested electromagnetic environments.

DARPA Awards Six Teams During Final Spectrum Collaboration Challenge Qualifier

“The six different scenarios were closely aligned to actual situations that our defense and commercial systems face in the field. The Wildfire scenario, for example, replicates the complex communications environment that surrounds an emergency response situation, while the Alleys of Austin scenario was designed to mimic what’s needed to help dismounted soldiers navigate and communicate as they sweep through an urban environment. This real-world relevance was critical for us as we want to ensure these technologies can continue to develop after the event and can transition to commercial and/or military applications,” said Tilghman.

The sixth scenario of the competition was used to determine the six prize winning teams. This scenario explored the essential question of the SC2 competition: can the top teams’ collaborative SC2 radios outperform the status quo of static allocation? Each of the six teams that received awards at PE2 demonstrated that their radio was capable of carrying more wireless applications without the aid of a handcrafted spectrum plan, while simultaneously ensuring four other radio networks operating in the same area had improved performance. In short, each of these six radio networks demonstrated the autonomous future of the spectrum.

To aid with decision making, teams applied AI and machine learning technologies in various ways. Some leveraged the current generation of AI technologies like deep learning, while others used more conventional optimization approaches. There were also a few teams that used first wave, rule-based AI technologies.

“We’re very encouraged by the results we saw at PE2. The teams’ radios faced new and unexpected scenarios but were still able to demonstrate smart, collaborative decision making. PE2 showed us that AI and machine learning’s application to wireless spectrum management creates a very real opportunity to rethink our current century-old approach,” said Tilghman.

DARPA NEXT GENERATION COMMUNICATIONS PROGRAM

The XG Program is developing technology and system concepts for military radios to dynamically access spectrum in order to establish and maintain communications. The goal is to demonstrate the ability to access 10 times more spectrum with near-zero setup time; simplify RF spectrum planning, management and coordination; and automatically de-conflict operational spectrum usage.

XG technology assesses the spectrum environment and dynamically uses spectrum across frequency, space and time. XG is designed to be successful in the face of jammers and without harmful interference to commercial, public service, and military communications systems. XG is transitioning to the Army to solve spectrum challenges in-theater.

In 2005, Shared Spectrum Company was awarded the prime contract to for Phase III of the neXt Generation Communications (XG) program funded by the Department of Defense’s (DoD) Defense Advanced Research Projects Agency (DARPA) and managed by the Air Force Research Laboratory (AFRL).

DOD’s Electromagnetic Spectrum Strategy

DoD’s growing requirements to gather, analyze, and share information rapidly; to control an increasing number of automated Intelligence, Surveillance, and Reconnaissance (ISR) assets; to command geographically dispersed and mobile forces to gain access into denied areas; and to “train as we fight” requires that DoD maintain sufficient spectrum access,” says DODS’s Electromagnetic Spectrum Strategy unveiled in February 2014

However, adversaries are aggressively developing and fielding electronic attack (EA) and cyberspace technologies that significantly reduce the ability of DoD to access the spectrum and conduct military operations. This requires development, fielding, and integration of complex EA, electronic support (ES), and electronic protection (EP) technologies to attack adversary’s command, control, communications, and computers; ISR; improvised explosive devices (IEDS); and area denial weapon systems, all of which require access to spectrum.

Concurrently, the unprecedented consumer demand for wireless mobility and data consumption has resulted in reduction and fragmentation of spectrum of defence. Only 1.4 percent of the RF spectrum from 0 to 300 GHz is available exclusively to the U.S. government. Additionally, the Defense Department also is under a mandate to give up 500 MHz of bandwidth for civilian use by 2020.

This has resulted in complex Defense, spectrum management within and between the armed services, and any errors in the spectrum management plan may result in the denial of critical strategic and tactical links. The second is relatively easy for adversaries to target such a small part of the RF spectrum allocated exclusively to the government through jamming or electronic attack.

DOD’s Electromagnetic Spectrum Strategy 2013 called for ensuring the access to the congested and contested electromagnetic environment of the future, by adopting new agile and opportunistic spectrum operations, and through systems which are more efficient, flexible and adaptable and adopting new technologies capable of more efficient use of the spectrum and reduced risk of interference.

The DoD EMS Strategy will focus on the following goals:

Advancing the spectrum-dependent technologies that are more efficient, flexible, and adaptable in their use of spectrum.

This will include:

Expediting development of technologies that increase a spectrum-dependent system’s ability to access wider frequency ranges, exploit spectrum efficiency gains, utilize less congested bands, and adapt to changing electromagnetic environments;
Pursuing spectrum sharing opportunities;
Evaluating commercial service capabilities (such as smartphones) for mission use; and
Improving DoD’s oversight of spectrum use.

Increasing the agility of DoD’s spectrum operations. This will include:

Managing spectrum-dependent systems in near-real-time by developing tools and techniques to quantify spectrum requirements and identify and mitigate spectrum issues;
Improving the ability to identify, predict, and mitigate harmful interference; and
Pursuing access to spectrum allocated for non-federal use and spectrum sharing technologies.

Encouraging DoD participation in changing national and international spectrum policy and regulation. In particular, DoD will focus on:

Developing innovative alternatives that consider both DoD and commercial interests; and
Improving its ability to adapt and implement regulatory and policy changes while maintaining full military capability

Opportunistic use of the spectrum is one of the promising approaches being pursued by both DoD and the wireless community. Therefore DoD systems must become more spectrally efficient, flexible, and adaptable, and DoD spectrum operations must become more agile in their ability to access spectrum in order to increase the options available to mission planners.

“DoD will also continue to adopt new tools and techniques to manage the spectrum more effectively, making our spectrum operations more agile,” says DOD’s spectrum strategy. Cognitive radio systems, improved spectrum sensing, and geo-location databases are among new opportunistic use technologies being considered.