Home / Technology / AI & IT / DARPA CASTLE developing AI-toolkit for network hardening against advanced persistent threats (APT)

DARPA CASTLE developing AI-toolkit for network hardening against advanced persistent threats (APT)

Computer networks are more than ever exposed to cyber threats of increasing frequency,
complexity and sophistication. Penetration testing (also known as pentesting or PT) is a common practice for actively assessing the defenses of a computer network by planning and executing all possible attacks to discover and exploit existing vulnerabilities.
In the current constantly evolving digital environment, PT is becoming a crucial and often mandatory component of cybersecurity auditing, particularly after the introduction of the European General Data Protection Regulation (GDPR) for organizations and businesses. In addition to legal requirements, PT is considered by the cybersecurity community as the most effective method to assess the strength of security defenses against skilled adversaries as well as the adherence to security policies
Pentesting (PT) is a non-standard active method for assessing network defenses by following
a sequential and interactive multi-phase procedure starting by gathering information and ending by reporting the obtained results. Research has investigated the possibility of automated tools for the different PT stages (reconnaissance, identification, and exploitation) to relieve the human expert from the burden of repetitive tasks.
However, automation by itself does not achieve many benefits in terms of time, resources and outputs because PT is a highly dynamic and interactive process of exploring and decision-making, which requires advanced and critical cognitive skills that are hard to duplicate through automation
A natural question arises in regard to the capability of AI to provide a potential solution that goes beyond simple automation to achieve expert-like output. AI has proven to be very helpful to not only offload work from humans but also possibly handle depths and details that humans can not tackle fast enough or accurately enough. Rapid progress in AI and notably the machine learning (ML) sub-field led us to believe that an AI-based PT system utilizing well-rounded models and algorithms for making sequential decisions in uncertain environments can bridge the gap between automation and expertise that PT community experience


The Defense Advanced Research Projects Agency (DARPA) Information Innovation Office (I2O) is sponsoring a Proposers Day to provide information to potential proposers on the objectives of the anticipated Cyber Agents for Security Testing and Learning Environments (CASTLE) Broad Agency Announcement (BAA). The CASTLE program seeks to develop an AI-toolkit to instantiate realistic network environments and train cyber agents to enable resilient network operations against advanced persistent threats (APT).


CASTLE will formulate network hardening as a reinforcement learning (RL) problem and train defensive agents in open, evolving, and adversarial environments that mimic actual networks. Environments will execute agents inside instrumented subnets that are deployed to live networks and will simulate defensive actions that counter APT tools.


Agent execution will produce calibrated datasets for progressively improving simulations. The CASTLE program aims to formulate network hardening as a reinforcement learning (RL) problem and teach RL agents to ‘operate through’ the post-breach behavior of widely available penetration testing tools. Over progressive rounds of attack and defense, agents will explore defensive actions to proactively stop on-going attacks while maintaining operationally relevant workflows. CASTLE workflows may encompass critical assets and essential services performed by networks.


The CASTLE program seeks to generate realistic environments that mimic actual networks. In these environments, agents will train to counter APT tools by learning automated defensive actions such as dataprotection policies, firewall rules, and device re-configurations. To support open and evolving training, environments and agents must allow progressive updates such as adding common vulnerabilities and exposures, ports, protocols, and services. Over the course of the program, CASTLE seeks to model networks at greater scale and fidelity and develop agents with more sophisticated defensive actions.

To improve simulations, CASTLE aims to instantiate environments as deployable subnets inside live networks, such that subnets are instrumented to record network and device events. Top performing simulated agents must be able to be instantiated inside subsets as well. Furthermore, the CASTLE program seeks to capture the side-effects of agent actions being executed inside the live network to inform future rounds of agent training. The CASTLE program views agent execution in instrumented subnets as a risk reduction measure intended to ensure simulations do not deviate from reality.

As an important benefit, instantiated environments and captured agent execution enable generation of labeled and continuously updated datasets. CASTLE aims to promote open, rigorous evaluations of defensive approaches by publicly releasing toolkit-generated datasets. Moreover, CASTLE aims for toolkit datasets to serve as standard benchmarks for rigorous measurement of cyber security performance beyond the program. Thus, proposers are encouraged to discuss concepts for publishing datasets to include labeling tool behavior, curating community-driven results, and making datasets amenable to open source machine learning libraries.

Since networks can differ greatly, the CASTLE program seeks research leading to open source standards and software for hardening networks. Indeed, the open source approach is expected to produce technology that is repeatable, portable, and shareable. Moreover, abstracting network hardening enables better collaboration between data scientists, machine learning researchers, and cybersecurity experts. Altogether, CASTLE aims to promote the adoption of a community-developed project that can contribute to collective network defense.

The Proposers Day will be held as a hybrid event on October 24, 2022. DARPA will host two in-person sessions in order to maximize the number of attendees who will participate in-person. Session 1 will be from 9:00AM to 12:00PM (ET), and Session 2 will be from 1:00PM to 4:00PM (ET). Both in-person sessions will be at the DARPA Conference Center, located at 675 N. Randolph Street, Arlington, Virginia, 22203. A virtual option for those who are unable to attend in-person is available through Zoom. Check-in for in-person attendees begins 8:30AM (ET) for Session 1 and 12:30PM for Session 2. Virtual attendees are encouraged to join Zoom ten minutes in advance of meeting start. Information on virtual check-in is available on the Proposers Day Website. Advance registration is required to attend in person.

Attendance at the CASTLE Proposers Day is voluntary and is not required to propose to subsequent solicitations (if any) on this topic. The Proposers Day does not constitute a formal solicitation for proposals. This notification is issued solely for information and program planning purposes, and is not a Request for Information (RFI). Since this is not an RFI, no submissions against this notice will be accepted by DARPA. DARPA will not provide reimbursement for costs incurred to participate in this Proposers Day. Interested parties to this notice are cautioned that nothing herein obligates DARPA to issue a solicitation.

About Rajesh Uppal

Check Also

Revolutionizing Real-Time Machine Learning: The Rise of Dedicated AI Chips and Neuromorphic Computing

Introduction: Artificial Intelligence (AI) has been a transformative force, aiming to enhance computers and robots …

error: Content is protected !!