The Defense Advanced Research Projects Agency (DARPA) has successfully created what it is calling a human memory prosthesis, a system with electrodes implanted in the brain that restores memory function using a person’s own neural codes.
In a DARPA-funded pilot study, scientists successfully tested what they call a “prosthetic memory” — a neural implant that can learn to recognize your brain activity when you correctly recall new information, and later replicate that activity with electrical signals to give your short-term memory a boost. In a small test of 15 patients at Wake Forest Baptist Medical Center, this prosthetic memory system helped the patients improve their short-term memory by an average of 35 percent. According to lead study author Robert Hampson, a professor of physiology, pharmacology and neurology at Wake Forest School of Medicine in North Carolina, this degree of short-term memory improvement is “huge.”
“We [had] the patient play a computer game that [involved] memory, and we [recorded] the activity of the brain cells — the neurons — in the hippocampus,” Hampson said. As the patients played, the researchers monitored their brain activity through the electrode implants. As patients answered more questions correctly, the researchers compiled an increasingly clear picture of what each patient’s mental activity looked like when their short-term memory was hard at work. During later trials, the researchers used these personalized memory codes to help stimulate specific parts of each patient’s brain. When patients received this mental stimulation, their recall improved.
“When we tested patients by stimulating their hippocampus with a pattern that was derived from their own neural activity… we were able to improve their short-term memory by quite a bit,” Hampson said.
DARPA has been working on restoring normal memory function in military personnel since 2013 in a program known as Restoring Active Memory (RAM). They’re getting real-world results in actual patients. Researchers from Wake Forest Baptist Medical Center and the University of Southern California implemented a proof-of-concept system that restored memory function by up to 37% improvement in short-term, working memory on baseline levels.
“DARPA’s cumulative investments in neurotechnology over the past two decades have brought us to an extremely exciting point today where we’re testing tangible technologies that have the potential to alleviate some of the worst effects of brain injury and illness,” said Justin Sanchez, the director of DARPA’s Biological Technologies Office and the program manager for RAM, in a press release.
DARPA’s “Restoring Active Memory” Program
Under BRAIN initiative, DARPA launched “Restoring Active Memory” in 2014, with the aim to develop wireless, fully implantable neural-interface medical devices that can serve as “neuroprosthetics”— devices that restore or supplement the mind’s capacities with electronics inserted directly into the nervous system. This technology can help veterans with Traumatic Brain Injuries (TBI), a serious cause of disability in the United States, diagnosed in more than 270,000 military service members since 2000 and affecting an estimated 1.7 million U.S. civilians each year.
Just over one year into the effort, the novel approach to facilitating memory formation and recall has already been tested in a few dozen human volunteers, said program manager Justin Sanchez. The study aims to give researchers the ability to “read” the neural processes involved in memory formation and retrieval, and even predict when a volunteer is about to make an error in recall. The implanted electrodes also provide a means of sending signals to specific groups of neurons, with the goal of influencing the accuracy of recall.
The new methods for analysis and decoding of neural signals are being explored, including the process by which newly learned information is attended to and processed by the brain when first encountered, and how neurons code knowledge about events, times, and places.
Initial results indicate that it is indeed possible to capture and interpret key signals or “neural codes” coming from the human brain during memory encoding and retrieval, and improve recall by providing targeted electrical stimulation of the brain.
Among other details, Sanchez said, the work is addressing the important issue of the ideal timing of electrical stimuli involved in the neural codes. “Should we provide electrical inputs when the lists are first being taught and memorized, or should we stimulate when the person is working to recall those items? We still have a lot to learn about how the human brain encodes declarative memory, but these early experiments are clarifying issues such as these and suggest there is great potential to help people with certain kinds of memory deficits,” Sanchez said.
New multi-scale computational models with high spatial and temporal resolution are being developed that will account for encoding and retrieval of complex memories and memory attributes, including their hierarchical associations with one another. This will allow Researchers to develop new, implantable, closed-loop systems that shall able to deliver targeted stimulation to help the brain reestablish an ability to encode new memories following brain injury.
The researchers on RAM focused on episodic memory which includes the short-term memory needed to remember where a car is, what someone had for dinner the night before or when they last took medication. Episodic memory loss is one of the most common types of memory loss in people who experience brain injury and Alzheimer’s disease.
The researchers focused on the hippocampus area of the brain, which is the area of the brain responsible for memory, to improve episodic memory. The hippocampus has sub-regions known as CA3 and CA1 that work together to support memory encoding and retrieval. When the brain wants to encode something as a memory, neurons are fired in the CA3 region, which inputs information into the hippocampus, and then neural activity occurs in the CA1 region which marks output from the hippocampus.
Epilepsy patients had electrodes surgically implanted in the CA3 and CA1 regions during the study to record neuronal activity. The participants then performed a visual memory test where they were shown an image like a color block and, after a brief delay, were asked to identify the image out of four to five other images.
As a result, the researchers could create a multi-input multi-output (MIMO) nonlinear, mathematical, spatiotemporal model to predict the transformation of neuronal firing patterns in the CA3 region into CA1 region firing patterns. The codes created by the model matched the CA1 firing patterns that happen when CA3 encodes information correctly, which corrects natural errors in the transformations between CA3 and CA1 firing patterns.
After the researchers created the firing pattern model, they performed another image-recall test on the volunteers using patterned, spatiotemporal electrical stimulation of CA1 to play back MIMO-based codes that were obtained from neuronal activity in CA3. The volunteers had an average of 37% improvement in episodic memory performance.
The researchers then tested their system to see if it would have a similar effect on long-term memory. The new test had highly-distinctive photographic images and the volunteers were asked to identify the photos out of a set of photos after a short delay.
Volunteers were also shown a set of three images after a 75-minute delay and were asked to identify the original photo from a set of four to five photos. The volunteers were stimulated with the correct spatiotemporal codes and had a 35% improvement in memory.
Stimulation did not replace CA1 activity in the brains during the study. Instead, the neural activity was supplemented by the stimulation.
“We’re closing in on our goal of an implantable, closed-loop memory prosthesis,” Sanchez said. “The RAM program continues to achieve and integrate amazing breakthroughs in neuroscience, artificial intelligence, and neural interface device development for clinical use, which makes me confident DARPA can ultimately deliver our Service members and veterans powerful technologies for countering memory loss.”
Restoring Active Memory Replay or RAM Replay
DARPA’s new effort is to develop neurotechologies that may help individuals not just better remember individual items but learn physical skills. Complex skills can take people years to master, and it’s not just repetition of the physical movements that matters. The process also often involves the repeated mental and physiological “replaying” of the skill during wakefulness and sleep to solidify the skill.
DARPA is seeking proposals to develop super learning and enhanced memory technologies for use in humans. Human memory is the result of biological processes that culminate in the formation or strengthening of neural connections in the brain. Multiple mechanisms are involved in memory formation and recall, including brain networks that govern perception, attention and emotion. Through a process known as consolidation, representations of experiences are stored in long-term memory and integrated with older knowledge and memories. DARPA seeks to expand the horizons of what is possible by “harnessing the brain’s own replay system to improve the strength and fidelity of memory”
The 24-month fundamental research program, Restoring Active Memory Replay or RAM Replay, is designed to develop novel and rigorous computational methods to help investigators determine not only which brain components matter in memory formation and recall but also how much they matter and the goal is to apply this knowledge to radically improve learning and memory. To ensure real-world relevance, those assessments will be validated through performance on DoD-relevant tasks instead of conventional computer-based behavioral paradigms commonly used to assess memory in laboratory settings.
“Military personnel carry a growing responsibility to recount, report and act upon knowledge gleaned from previous experiences, and how well those experiences are recalled can make all the difference in how well these individuals perform in combat and other challenging situations,” said Dr. Justin Sanchez, DARPA program manager.
“In the long run, we hope RAM Replay will identify core memory-strengthening mechanisms and give rise to a generalizable set of solutions applicable to the challenge of memory reliability in an increasingly information-dense world. That could benefit civilians and Service members alike in areas as diverse as general education, job retraining and battlefield awareness.”
US Army successfully reads soldier’s brain signals
The U.S. Army has successfully used a desktop computer to analyse the brain waves of the soldier to determine what target image a soldier was thinking about. The experiment took place at an Army research facility known as “The MIND Lab”, where the soldier sat in front of a computer looking at images that flashed up on screen, at a rate of about one per second, according to a statement released by the Army. The soldier’s brain signals were read by electroencephalogram. There were five categories of images – boats, pandas, strawberries, butterflies and chandeliers.
The soldier was asked to choose one of the categories, but keep the choice to himself. “When the experiment was over, after about two minutes, the computer revealed that the Soldier had chosen to focus on the ‘boat’ category,” explained the Army, in its statement, noting that the soldier’s brain waves were analyzed.
Cognitive neuroscientist Anthony Ries said that the technology could be useful to the intelligence community, which often has to analyze vast quantities of image data. Brain waves, or “neural signals” could be used to quickly identify targets of interest, he explained. “What we are doing is basically leveraging the neural responses of the visual system,” Ries noted, in the statement. “Our brain is a much faster image processor than any computer is. And it’s better at detecting subtle differences in an image.”
Mind copying or Mind transfer
The understanding of memory formation, reactivation, and recall mechanisms developed in this program are important transhumanist projects such as whole brain emulation and mind uploading. Whole brain emulation (WBE) or mind uploading (sometimes called “mind copying” or “mind transfer”) is the hypothetical process of copying mental content (including long-term memory and “self”) from a particular brain substrate and copying it to a computational device, such as a digital, analog, quantum-based or software-based artificial neural network.
The computational device could then run a simulation model of the brain information processing, such that it responds in essentially the same way as the original brain (i.e., indistinguishable from the brain for all relevant purposes) and experiences having a conscious mind. Among some futurists and within the transhumanist movement, mind uploading is treated as an important proposed life extension technology
Future Brain implantable chips could help regain long term memory
Ted Berger, a biomedical engineer and neuroscientist at the University of Southern California, has been able to model the transformation of information from the prefrontal cortex of brain responsible for the processing of short term memory, to the hippocampus responsible for long term memory processing.
Through programmed external chips, they were able to regain memories in mice and monkeys. Berger’s research proves that in future implantable brain chips could be used to mimic brain functions, and soon, may allow chips to introduce skills and knowledge into the brain.
Systems-Based Neurotechnology for Emerging Therapies (Subnets)
DARPA’s $70 million project called the Systems-Based Neurotechnology for Emerging Therapies (Subnets), is inspired by Deep Brain Stimulation, or DBS, a surgical treatment that involves implanting a brain pacemaker in the patient’s skull to interfere with brain activity and help with symptoms of diseases like epilepsy and Parkinson’s. DARPA’s device will be similar, but rather than targeting one specific symptom, it will be able to monitor and analyze data in real time and issue a specific intervention according to brain activity.
Through measuring pathways involved in complex systems-based brain disorders including post-traumatic stress, major depression, borderline personality, general anxiety, traumatic brain injury, substance abuse and addiction, and fibromyalgia/chronic pain, SUBNETS will pursue the capability to record and model how these systems function in both normal and abnormal conditions, among volunteers seeking treatment for unrelated neurologic disorders and impaired clinical research participants.
SUBNETS will then use these models to determine safe and effective therapeutic stimulation methodologies. These models will be adapted onto next-generation, closed-loop neural stimulators that exceed currently developed capacities for simultaneous stimulation and recording, with the goal of providing investigators and clinicians an unprecedented ability to record, analyze, and stimulate multiple brain regions for therapeutic purposes.
Just one year into the SUBNETS effort, engineers at the Lawrence Livermore National Laboratory and Draper Laboratory have produced customized electrode arrays and miniaturized neural interface hardware. The prototypes include microfabricated electrode arrays that are flexible and can interface with large numbers of neurons; fully implantable hardware to amplify and interpret brain signals; and new circuitry to deliver precise, function-restoring feedback to the brain.
In the first clinical tests of some of these technologies, researchers at the University of California, San Francisco, placed arrays on the brains of seven patients and, by providing electrical impulses to a specific neuronal region, markedly reduced the patients’ anxiety levels.
“We’re talking about a whole systems approach to the brain, not a disease-by-disease examination of a single process or a subset of processes,” DARPA program manager Sanchez said. “Subnets is going to be a cross-disciplinary, expansive team effort, and the program will integrate and build upon historical DARPA research investments.”
Deep Brain Stimulation, or DBS, requires precise knowledge of regions of brain where electrodes are required to be placed. Michele Tagliati’s group from the Movement Disorders Program in the department of neurology at Cedars-Sinai have created 3D anatomical models of several patient brains from MRI scans to aid in the placement of DBS electrodes. Using multiphysics simulations they were then able input actual stimulation parameters to optimize the effects.
Allan Snyder, professor of neuroscience in Australia, has been working to understand how transcranial magnetic stimulation (TMS) can effect higher mental processing with the use of magnetic fields to promote unfettered reasoning. Researchers from ASU, have engineered a novel technology, which implements transcranial pulsed ultrasound to remotely and directly stimulate brain circuits without requiring surgery. This could be implanted in troops’ battle helmets, allowing soldiers to manipulate brain functions to boost alertness, relieve stress, or even reduce the effects of traumatic brain injury.
Manipulating the brain to enhance warfighting capabilities and maintain mental acuity on the battlefield has long been a topic of interest for DARPA and various military research labs. This technology can be used to produce super soldiers, could be used in battlefields to expand a soldier’s technical expertise and become a more proficient marksman by introducing new skills and knowledge into the brain.
Mark Goodman, former FBI and secret service employee has warned that the rise of this type of technology could lead to brain hackers. According to Goodman, brain implants may not even be necessary for someone to erase your memory or read your mind, but a microchip would certainly make it a lot easier.
“I think holding people’s memories hostage in demand of an extortion payment would be a fairly horrific crime. Of course, this is all theoretical now. We do not have the capacity to do this. But I thoroughly believe it will be forthcoming. I think we’ll see more of that, which could lead to all kinds of problems. Forget memorizing passwords—somebody could just pull that data from a brain scan,” said Goodman.
“Once organized crime figures out how to do this, I could see a horrific scenario wherein somebody is kidnapped and threatened that unless they pay an exorbitant sum, their lifelong memories of their wife or daughter would be erased,” he added.
President Obama announced “Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative” in April 2013 to support and coordinate research on how the brain functions over an organism’s lifespan. The BRAIN Initiative is intended to accelerate the development of experimental and theoretical approaches and novel neurotechnologies essential for understanding the dynamics and principles of brain structure and function.
This multi-agency Initiative is led by NSF, along with the National Institutes of Health (NIH) and the Defense Advanced Research Projects Agency (DARPA) and includes private partners.
DARPA Director Arati Prabhakar said that “DARPA hopes to build tools that can view, measure and control the brain from the cellular and neuronal to the macroscopic levels”. “We’re driving toward clearer models of how memory is encoded in the brain and how lesions that lead to memory loss can be circumvented in order to restore memory that may have been damaged due to post traumatic stress or brain injury”, according to Prabhakar.
This research has already triggered many ethical concerns about their development and implementation. Apart from the inherent risks to patients’ physical wellbeing, memory is a fundamental part of a person’s identity, and any mistake could erase part of what makes a person unique.
The chips can also erase unwanted memories. By eliminating empathy, the Department of Defense (DoD) hopes to “enhance” a soldier’s ability to “kill without care or remorse, shows no fear, can fight battle after battle without fatigue and generally behave more like a machine than a man. Ultimately the technology can also be utilized to control soldiers’ brains through the chips