DARPA’s RTML to automatically create novel chip designs of Application-Specific Integrated Circuits (ASICs) for defense machine learning systems

A grand challenge in computing is the creation of machines that can proactively interpret and learn from data in real time, solve unfamiliar problems using what they have learned, and operate with the energy efficiency of the human brain. While complex machine-learning algorithms and advanced electronic hardware  that can support large-scale learning have been realized in recent years and support applications such as speech recognition and computer vision, emerging computing challenges require real-time learning, prediction, and automated decision-making in diverse domains such as autonomous vehicles, military applications, healthcare informatics and business analytics.

Driven by the rapidly evolving national security threat landscape, future defense systems will need access to low size, weight, and power (SWaP) artificial intelligence (AI) solutions that can rapidly transition from idea to practice. In recent years, the ability to learn from large datasets has advanced significantly due to increases in hardware performance, advances in machine learning (ML) algorithms, and the availability of high quality open datasets. However, current ML systems are generally trained prior to deployment and are not capable of adapting to new datasets in the field, limiting real-time “learning.”

The current generation of machine learning (ML) systems would not have been possible without significant computing advances made over the past few decades. The development of the graphics-processing unit (GPU) was critical to the advancement of ML as it provided new levels of compute power needed for ML systems to process and train on large data sets. As the field of artificial intelligence looks towards advancing beyond today’s ML capabilities, pushing into the realms of “learning” in real-time, new levels of computing are required. Highly specialized Application-Specific Integrated Circuits (ASICs) show promise in meeting the physical size, weight, and power (SWaP) requirements of advanced ML applications, such as autonomous systems and 5G. However, the high cost of design and implementation has made the development of ML-specific ASICs impractical for all but the highest volume applications.

Critical next-generation defense systems, such as autonomous vehicles and arrays of sensors, will be deployed in distributed settings where resources for exporting newly encountered data might be scarce or unavailable. The Real Time Machine Learning (RTML) program seeks to solve this problem by creating no-human-in-the-loop hardware generators and compilers to enable the fully automated creation of ML Application-Specific Integrated Circuits (ASICs) from high-level source code. The RTML program is focused on building an end-to-end general purpose compiler that can transform a high level ML framework into Verilog. This, in turn, may allow future engineers to rapidly develop and deploy large-scale real-time machine learning systems with customized hardware that can execute intensive ML algorithmic tasks on chip, without the need for external computational resources.

“A critical challenge in computing is the creation of processors that can proactively interpret and learn from data in real-time, apply previous knowledge to solve unfamiliar problems, and operate with the energy efficiency of the human brain,” said Andreas Olofsson, a program manager in DARPA’s Microsystems Technology Office (MTO). “Competing challenges of low-SWaP, low-latency, and adaptability require the development of novel algorithms and circuits specifically for real-time machine learning. What’s needed is the rapid development of energy efficient hardware and ML architectures that can learn from a continuous stream of new data in real time.”

DARPA’s Real Time Machine Learning (RTML) program seeks to reduce the design costs associated with developing ASICs tailored for emerging ML applications by developing a means of automatically generating novel chip designs based on ML frameworks. The goal of the RTML program is to create a compiler – or software platform – that can ingest ML frameworks like TensorFlow and Pytorch and, based on the objectives of the specific ML algorithms or systems, generate hardware design configurations and standard Verilog code optimized for the specific need. Throughout the lifetime of the program, RTML will explore the compiler’s capabilities across two critical, high-bandwidth application areas: 5G networks and image processing.

“Machine learning experts are proficient in developing algorithms but have little to no knowledge of chip design. Conversely, chip designers are not equipped with the expertise needed to inform the design of ML-specific ASICs. RTML seeks to merge these unique areas of expertise, making the process of designing ultra-specialized ASICs more efficient and cost-effective,” said Olofsson.

Based on the application space’s anticipated agility and efficiency, the RTML compiler provides an ideal platform for prototyping and testing fundamental ML research ideas that require novel chip designs. As such, DARPA plans to collaborate with the National Science Foundation (NSF) on this effort. NSF is pursuing its own Real Time Machine Learning program focused on developing novel ML paradigms and architectures that can support real-time inference and rapid learning. After the first phase of the DARPA RTML program, the agency plans to make its compiler available to NSF researchers to provide a platform for evaluating their proposed ML algorithms and architectures. During the second phase of the program, DARPA researchers will have an opportunity to evaluate the compiler’s performance and capabilities using the results generated by NSF. The overall expectation of the DARPA-NSF partnership is to lay the foundation for next-generation co-design of RTML algorithms and hardware.

“We are excited to work with DARPA to fund research teams to address the emerging challenges for real-time learning, prediction, and automated decision-making,” said Jim Kurose, NSF’s head for Computer and Information Science and Engineering. “This collaboration is in alignment with the American AI Initiative and is critically important to maintaining American leadership in technology and innovation. It will contribute to advances for sustainable energy and water systems, healthcare logistics and delivery, and advanced manufacturing.”

RTML is part of the second phase of DARPA’s Electronics Resurgence Initiative (ERI) – a five-year, upwards of $1.5 billion investment in the future of domestic, U.S. government, and defense electronics systems. As a part of ERI Phase II, DARPA is supporting domestic manufacturing options and enabling the development of differentiated capabilities for diverse needs. RTML is helping to fulfill this mission by creating a means of expeditiously and cost-effectively generating novel chip designs to support emerging ML applications.

DARPA award to University of Minnesota

The University of Minnesota Twin Cities recently received a $2.2 million grant from the Defense Advanced Research Projects Agency (DARPA), an agency of the U.S. Department of Defense, to build open-source hardware generators for a range of machine learning algorithms that process data in real time. The project is being funded under the Real Time Machine Learning program. ECE’s Prof. Sachin Sapatnekar will lead the project, and will collaborate with professors Hadi Esmaeilzadeh and Andrew B. Kahng of University of California San Diego, and Prof. Jie Gu of Northwestern University. The latter is an ECE alumnus, having earned his doctoral degree in 2008.

Critical improvements in computing technology in recent decades have enabled the current generation of machine learning. The graphics processing unit (GPU), for instance, has provided an altogether new level of computing power that has allowed machine learning systems to process large data sets. With artificial intelligence moving towards real time learning, current machine learning capabilities must be further advanced. Application Specific Integrated Circuits (ASICs) have the potential to meet the needs of advanced machine learning applications in an energy-efficient manner. Currently, however, the costs associated with developing such integrated circuits are prohibitive.

The RTML program seeks to develop low-cost Application Specific Integrated Circuits (ASIC) for emerging machine learning applications. The goal is to develop a compiler that, based on the objectives of the machine learning algorithm, can automatically generate hardware design configurations and standard Verilog code that can address specific needs. Two vital high-bandwidth application areas that the RTML program is targeting are 5G networks and image processing.

The RTML undertaking is the second part of DARPA’s Electronics Resurgence Initiative (ERI), investing more than $1.5 billion in advancing domestic, national, and defense electronic systems. In this phase, the agency is supporting domestic manufacturing options, and the development of circuits capable of meeting diverse and advanced needs. To this end, the RTML program seeks to develop ways in which novel chip designs can be created quickly and at low costs to support emerging machine learning applications.