DARPA’s CHASE to automate cybersecurity across the large, distributed Department of Defense, Enterprise Network

Rajesh Uppal January 26, 2019 Cyber, Industry Comments Off on DARPA’s CHASE to automate cybersecurity across the large, distributed Department of Defense, Enterprise Network 1,439 Views

Secure networks are vital to U.S. military operations at all levels and are a source of tactical, operational and strategic advantages. They provide the capability to acquire, move and process massive amounts of information, make decisions rapidly and command distributed forces on an unparalleled scale.

However, Networks within the United States and abroad face increasingly broad-spectrum cyber threats from numerous actors and novel attack vectors. DoD networks are under continuous attack, 250,000 a day by some estimates, ranging from curious teens to the advanced persistent threat and malicious insiders.

The DoD is not only threatened by attacks on its own networks but by those against government agencies with whom the Pentagon works, the private sector companies from whom it depends for a vast array of goods and services and portions of the nation’s critical infrastructure on whom it is dependent. Malicious activity also crosscuts organizational boundaries, as nefarious actors use networks with less protection to pivot into networks containing key assets.

U.S. Army “Shaping the Army Network: 2025-2040,” vision for an effective, modern enterprise network will let soldiers fight in joint, interagency and multinational environments. Its vision identifies five areas where the service needs leap-ahead technologies and network capabilities: dynamic transport, computing and edge sensors; data to decisive action; human cognitive enhancement; robotics and autonomous operations; and cybersecurity and resiliency. There is no value to investing in the best network technologies if they are vulnerable to attack. Success in future conflicts will go to the side best able to defend their networks from penetration, exploitation and attack.

In order to more effectively, efficiently and securely exploit the advantages inherent in a networked force, the Department of Defense (DoD) is beginning to change the way it organizes and manages its networks. The Pentagon is building the Joint Information Environment (JIE), a single joint enterprise IT platform that can be leveraged for all DoD missions. It is designed to provide greater standardization, economies of scale, end-to-end visibility and new, single security architecture. The JIE envisions a single security architecture that will provide the organizational backbone, operational coherence, end-to-end situational awareness (SA) and rapid response needed to provide cyber security for the massive and growing IT environment.

Enterprise-sized networks like JIE present challenges in terms of their size and distributed structure. What makes it even more challenging is not knowing where, when, and how enterprise network attacks will occur and what kind of tactics attackers are using. The Common Vulnerabilities and Exposures system, which tracks cyber security vulnerabilities, records about 7,000 new exploits annually. However, in 2017 that number jumped to more than 14,000, Roberts noted.

Detection of these threats requires adjustments to network and host sensors at machine speed. Additionally, the data required to detect these threats may be distributed across devices and networks. In all of these cases, the threat actors are using technology to perpetrate their attacks and hide their activities and movement, both physical and virtual, inside DoD, commercial, and Internet Access Provider (IAP) networks. Today’s state-of-the-art commercial tools do not directly address the scale and speed needed to provide the best defence for multiple networks, according to DARPA. The US Defense Advanced Research Projects Agency (DARPA) is seeking to improve how enterprise networks can rapidly detect and defend against cyber attacks.

The program, called Cyber Hunting at Scale (CHASE), uses computer automation, advanced algorithms and a new caliber of processing speed to track large volumes of data in real-time, enabling human cyber hunters to find advanced attacks otherwise hidden or buried within massive amounts of incoming data. Working in tandem with DARPA, a BAE Systems scientist says the potential promise of these advanced techniques is quite significant, because there is often simply not enough storage and memory to monitor nearly 80-percent of trafficking data goes undetected in large enterprise networks.

“Cyber hunt teams are currently massively overburdened and can only look at a small percentage of data collected using filters. Advanced adversaries take advantage of this,” Sam Hamilton, BAE Systems Chief Scientist, told Warrior Maven in an interview earlier this year. “Sophisticated adversaries understand today’s cyber defense chain very well and are building things to defeat it.”

Hamilton further specified that increasingly sophisticated adversaries are developing methods of hiding attack “footprints,” or weaving them into data streams not likely to be flagged at high-priority by cyber defenders. CHASE uses “adversary resistant” machine learning, developers explain; the aim of machine-learning is to build automation able to organize and analyze new information by identifying patterns, placing things in context and comparing new data against very large historical databases.

The Defense Advanced Research Projects Agency is seeking tools that can coordinate cybersecurity across the large, distributed networks of the Department of Defense. The goal is to be able to build adaptive technologies that can recognise, detect, and defend enterprise networks with the same kind of speed and agility that cyber attacks have, Dr Jennifer Roberts, programme manager for DARPA’s Cyber-Hunting at Scale (CHASE) programme, told Jane’s .

The concept is to not only thwart commonly used malware, phishing and denial-of service attacks but also defeat much more elaborate, sophisticated kinds of attacks. “An advanced piece of malware could be a program designed to hide in computer memory or on a router,” Hamilton explained.

DARPA’s Cyber-Hunting at Scale (CHASE) programme

Traditionally, cyber defense technologies focus predominantly upon either host data or network data. Malicious activity, however, crosscuts networks and hosts. Real-time detection of threats within or across very large enterprise networks is not simply an issue of scale, but also a challenge due to the variable nature of malicious activities and their presentations. Networks lack robust mechanisms to collect, share, and respond to threat intelligence. Data required to detect and characterize malicious activities may be diffused and may be located across network and endpoint devices.

Further, cyber-relevant data (including data that may contain information useful for detection and characterization) routinely exceeds total available storage, bandwidth, and analysis capability, often by several orders of magnitude. Of data that is able to be stored, only some is currently analyzed, and of all alerts generated, only a fraction is threat related.

“If you look at cyber defense across multiple enterprise networks, what you’ll find very quickly is maybe on the order of a dozen networks, and the folks who are using those networks already generating cyber defense-relevant data. But then, if you aggregate the amount of data storage that we [need] in order to process that data and detect who is attacking our networks, we have a fraction of that. Often it’s on the order of 100 petabytes aggregate, and it depends on the network. That means the cyber-relevant data that can tell us who’s attacking the network is an order of magnitude larger than our storage capabilities every single month. So we can’t catch up,” said Jennifer Roberts, the CHASE program manager .

We also have a constraint where algorithmically we can only process a very small fraction of the data in front of a human analyst. The essence of the program is how do you get the right data from the right device at the right time in order to really bolster our security in our networks.

Storage and processing limitations abound, so cyber defenders require tools that strategically direct resources toward the data that actually contains information about threats. Current commercially available tools may output thousands of alerts and false positives per day that often cannot be verified due to a lack of processing capacity. Static data retention policies sometimes result in the deletion of relevant data prior to investigation. Additionally, current tools may neither proactively detect novel attack vectors nor detect coordinated attacks distributed across multiple organizations.

The CHASE program seeks to develop automated tools to detect and characterize novel attack vectors, collect the right contextual data, and disseminate protective measures both within and across enterprises. CHASE aims to prototype components that enable network owners to reconfigure sensors and disseminate protective measures at machine speed with appropriate levels of human supervision. CHASE technologies will explore real-time investigations of potential cyber threats through adaptive data collection.

The program has five technical areas — threat detection and characterization; informed data planning; global analysis; protective measure generation and dissemination; and infrastructure for evaluation exercises — with multiple awards available in the first four sectors. A single contract will be offered for the fifth area.

Threat detection algorithms developed under CHASE may be tailored to characterize and react to specific classes of threats in the context of different data types and data sources. Additionally, these algorithms may work in concert to determine probabilities of the reality of threats, as well as indicate requirements for additional data that should be collected. As such, the goal of CHASE is to develop foundational technologies for detection, characterization, and strategic data management. Enhanced threat detection may cue the generation of automated protective measures. CHASE will focus on protective measures that a network owner has the authority to execute within their own environment, as well as measuring the accuracy and efficiency of threat detection techniques.

Roberts hopes to develop data-driven methods to better protect and defend these networks. She is currently in the midst of contract negotiations with the CHASE participants and expects the programme to begin by mid-2018.

CHASE will be broken down into four technical areas, with the first to focus on detection and characterisation of malicious activity. The objective is to create new data-driven algorithms to prioritise likely malicious activity occurring in the network and where cyber analysts should be investigating and spending their time.

While there are technologies that will generate alerts when something that might be indicative of malicious activity is occurring within the network, teams assigned to defend those networks could receive thousands of alters per day and often they are not prioritised, Roberts said.

The second technical area is exploring adaptive sensor technologies that could help operators understand if the movement of large amounts of files is someone stealing the company’s intellectual property or just downloading a movie.

The third area is taking more of a global perspective by detecting certain types of attacks that are only visible when looking across multiple enterprises. For example, those carried out by criminal networks or nation-states, Roberts said.

“This technical area is looking for new types of algorithmic techniques to really give us early warnings and indicators of more global types of attacks,” she said. “Being able to detect that type of progression of attack can really help us bolster the overall security across many networks.”

The fourth technical area is looking at automated generation and dissemination of protected measures, such as automatically updating firewalls or isolating a device from the network.

Having an indication that someone is stealing data could trigger an automated mechanism that blocks any data from leaving the network or isolates it until additional information can be gathered, Roberts said.

DARPA is focused on R&D…. But in order to get to where we have some type of platform that’s deployed to hundreds of parts of government networks, that requires investments from the agencies that are building platform and deployment technology and providing the long-term maintenance.

What we end up doing is forming partnerships with the organizations that might shelter the technology long-term. And during the course of the program we’re working to establish these partnerships with the folks that will be responsible for maintenance and the second half of development, working alongside the researchers, so we can really build the technologies to be used by many folks … and make it easier for analysts to protect their networks.

DARPA awards

Perspecta Inc. announced its research arm Perspecta Labs was awarded a prime contract from the Defense Advanced Research Projects Agency to support the Cyber-Hunting at Scale, or CHASE, program. The goal of the CHASE program is to develop dynamic, real-time tools that can successfully defend large-scale, distributed networks from cyber threats.

The 4-year contract, which has a base value of $4.7 million, calls for Perspecta Labs to develop, demonstrate and evaluate data-driven real-time cyber-hunting tools. Specifically, the company plans to design and develop WILEE, a set of components for detecting threats. The company said its WILEE solution will accelerate the hunt process by translating high-level threat descriptions into concrete implementation using a variety of sophisticated analytic techniques.

“This new work builds on Perspecta Labs’ research experience, technology expertise and implementation capabilities on past cyber work with DARPA,” said Petros Mouchtaris, president of Perspecta Labs. “We look forward to working closely with DARPA on this critical work to harden distributed networks.”

BAE Systems to develop cyber defence tools for DARPA’s CHASE programme

The US Defense Advanced Research Projects Agency (DARPA) has selected BAE Systems to develop data-driven, cyber-hunting tools that detect and analyze cyber threats to help protect extremely large enterprise networks.

Because most current tools do not offer the scale and processing speed needed to adequately defend enterprise networks, the goal of DARPA’s Cyber-Hunting at Scale (CHASE) program is to develop, demonstrate, and evaluate new, automated cyber-defense tools for use within and across these types of networks. BAE Systems’ unique solution, which combines advanced machine learning and cyber-attack modeling, intends to address this critical need by automatically detecting and defeating advanced cyber threats that could currently go undetected. The result could be better-defended commercial networks, using existing storage and existing resources. The technology could also be used to help protect government and military networks.

“Today, advanced cyber attacks within many enterprise networks go entirely unnoticed among an overwhelming amount of network data, or they require intensive manual analysis by expert teams,” said Anne Taylor, product line director for the Cyber Technology group at BAE Systems. “Our technology aims to alleviate resource constraints to actively hunt for cyber threats that evade security measures, enhancing the collective cyber defense of these networks”.

Perspecta Wins Cyber Hunting Contract with DARPA: Oct 2018

Perspecta Inc. (NYSE: PRSP) announced today that its innovative research arm, Perspecta Labs, was awarded a prime contract from the U.S. Defense Advanced Research Projects Agency (DARPA) to support the Cyber-Hunting at Scale (CHASE) program. The four-year contract which represents new work for the company has a base value of $4.7M, not including options.

On the contract, Perspecta Labs will develop, demonstrate and evaluate data-driven cyber-hunting tools that work in real-time, at-scale, and across multiple enterprise networks. Specifically, the company plans to design and develop WILEE, a set of components for threat detection and characterization.

The Perspecta Labs’ WILEE solution will accelerate the hunt process by translating high-level threat descriptions into possible concrete implementations using a variety of sophisticated analytic techniques, including adversarial planning, genetic perturbation and data-driven evaluation. WILEE will automatically prioritize hunt activities, inform data collection, minimize detection time and detect both known and novel malicious activities with high confidence.