The brain-computer interface (BCI) allows people to use their thoughts to control not only themselves, but the world around them. Every action our body performs begins with a thought, and with every thought comes an electrical signal. The electrical signals can be received by the brain-computer interface, through an electroencephalograph (EEG) or an implanted electrode, which can then be translated, and then sent to the performing hardware to produce the desired action.
Brain-computer interfaces are being applied in neuroprosthetics, through which paralyzed persons are able to control robotic arms, neurogaming where one can control keyboard, mouse etc using their thoughts and play games, neuroanalysis (psychology), and now in military and defense to control robotic soldiers or fly planes with thoughts. The obvious application for the military is mind-controlled weaponry and remotely-piloted aircraft, which could make operation and reactions far faster.
Russian scientists have developed the first electric car in the world that will be controlled by brain through an innovative system of direction by ‘telekinesis’ that will allow the exchange of information between the brain and the vehicle’s control systems. The ‘neuromobile’, as its creators call it, will allow people with limited mobility to cover long distances without help.
Eventually, brain-computer interfaces could let people control augmented reality and virtual reality experiences with their mind instead of a screen or controller. Facebook’s CEO and CTO teased these details of this “direct brain interface” technology over the last two days at F8.
Brian Control Interfaces
Facebook revealed it has a team of 60 engineers working on building a braincomputer interface that will let you type with just your mind without invasive implants. The team plans to use optical imaging to scan your brain a hundred times per second to detect you speaking silently in your head, and translate it into text. Facebook tells Josh Constine “This isn’t about decoding random thoughts. This is about decoding the words you’ve already decided to share by sending them to the speech center of your brain. Regina Dugan, the head of Facebook’s R&D division Building 8, explained to conference attendees that the goal is to eventually allow people to type at 100 words per minute, 5X faster than typing on a phone, with just your mind.
“Our brains produce enough data to stream 4 HD movies every second. The problem that the best way we have to get information out into the world – speech – can only transmit about the same amount of data as a 1980s modem. We’re working on a system that will let you type straight from your brain about 5x faster than you can type on your phone today,” said Mark Zuckerberg.
Earlier this year, in collaboration with Johns Hopkins Medicine, APL demonstrated the ability to decode semantic information — information about the meanings of words — from neural signals measured using electrodes placed on the surface of the brain in patients undergoing treatment for epilepsy. Similarly, APL has been designing noninvasive optical imaging methods to replace the use of implanted electrodes in order to make these technologies accessible beyond clinical applications. APL is working on project that focuses on developing a silent speech interface that will allow users to type 100 words per minute — five times faster than typing on a smartphone — using only their thoughts. “The research agreement with Facebook has also allowed us to expand our pioneering brain–machine interface work, and further combine our expertise in neuroscience with our expertise in optical imaging.”
Researchers recorded ECoG while patients named objects from 12 different semantic categories, such as animals, foods and vehicles. “By learning the relationship between the semantic attributes associated with objects and the neural activity recorded when patients named these objects, we found that new objects could be decoded with very high accuracies,” said Michael Wolmetz, a cognitive neuroscientist at the Johns Hopkins Applied Physics Laboratory, and one of the paper’s authors. “Using these methods, we observed how different semantic dimensions — whether an object is manmade or natural, how large it typically is, whether it’s edible, for example — were organized in each person’s brain.”
Russian, Chinese and German Researchers develop cars controlled by mind alone
The mind controlled full-scale electric car has been designed by specialists from Russian university N.E. Lobachevsky situated in Nizhniy Novgorod. The creators of the prototype said that they had to develop a system for recording brain signals of different modalities and special algorithms to allow the car to decipher commands. The most innovative part of the vehicle, is its control system, as it includes the use of special algorithms capable of reading different signal modes and transferring them to the machine’s control system .
Specifically, the algorithms will classify “a set of data on the physical state of the person” received by the control system to “discern the mental instructions of the pilot”, as explained at the university, reports RG. In other words, the system “will determine the variant of maneuver in a situation of circulation in which the driver thought”. Then the driver’s mental commands will be transmitted to the actuators of the car’s control system.
The developers believe that the unique car will go into mass production within three years and will be used primarily by handicapped citizens. The creators also stressed that the Russian neuro-mobile will prove to be not only advanced in terms of technology, but will also be economical and affordable.
“A significant advantage is the low cost 550 000 — 990 000 rubles [$9000 to $16,000] while the purchase of foreign cars with similar characteristics will cost 50%
Researchers at Nankai University in Tianjin, China are working alongside Chinese Automaker Great Wall Motor to design a car which can be controlled by the mind alone. During the test the subject using a 16 sensor headset vehicle, was able to command the car to accelerate, break and open and shut the doors.
“There are two starting points of this project. The first one is to provide a driving method without using hands or feet for the disabled who are unable to move freely; and secondly, to provide healthy people with a new and more intellectualized driving mode,” researcher Zhang Zhao told Reuters.
According to the researchers, the ultimate plan could be to integrate this technology with driverless cars, so it is more of a complementary service than an alternative to physical driving. Professor Duan Feng, who led the project, told Reuters, “In the end, cars, whether driverless or not, and machines are serving for people. Under such circumstances, people’s intentions must be recognized. In our project, it makes the cars better serve human beings.
A team of researchers at the Free University of Berlin has also explored brain interfaces to steer vehicles. The German-based team, led by artificial lab professor Dr. Raul Rojas, used a headset and electroencephalography (EEG) sensors designed by bioinformatics company Emotiv. The system was able to interpret the driver’s thoughts such as desire to turn left, right, accelerate and brake, and create computer commands.
DARPA’s Mind Controlled Prosthetic Arm
DARPA’s Prosthetic Arm is set to take over a real arm, letting the receiver control it with their thoughts, fired up by brain cells. The robotic arm is connected by wires that link up to the wearer’s motor cortex—the part of the brain that controls muscle movement—and sensory cortex, which identifies tactile sensations when you touch things. In essence, they claim, it allows its user to feel things with their robotic hand.
In their research, the HAPTIX team is implanting electrodes in a patient’s muscles between the elbow and shoulder, as well as in individual nerve fascicles that correspond to wrist and finger control. According to the release, the researchers are also looking to develop minimally invasive procedures to implant electrodes in the spinal cord. The HAPTIX researchers seek to acquire and decode neural signals that could provide intuitive prosthetic control and restore sensory feedback using these neural interface systems.
“We want to re-establish communication between the motor parts of the nervous system and the prosthetic hand through the use of implantable electronics,” Weber said in a press release.
The HAPTIX program is in its second phase, which is scheduled to continue through 2018. The third phase is scheduled for 2019, when transradial amputees will be allowed to take home a HAPTIX-controlled system for extended trials outside the laboratory, the press release noted.
Researchers at the University of Pittsburgh were able to increase the maneuverability of the mind controlled robotic arm from seven dimensions (7D) to 10 dimensions (10D). The extra dimensions come from four hand movements–finger abduction, a scoop, thumb extension and a pinch—enabling one to pick up, grasp and move a range of objects much more precisely than with the previous 7D control. This in turn help paralyzed persons to control a robotic arm with a range of complex hand movements.
“The ultimate goal for HAPTIX is to create a device that is safe, effective, and reliable enough for use in everyday activities,” explains Doug Weber, the DARPA HAPTIX program manager.
California Institute of Technology’s BCI allow Paralyzed Man to Drink Beer on his own
A paralyzed man named Erik Sorto has been able to drink beer, shake hands and even play “rock, paper and scissors,” thanks to a robotic arm controlled solely by his mind.
For this experiment California Institute of Technology neuroscientist Richard Andersen implanted the electrodes of BCI in different area of the brain, the posterior parietal cortex, which is located on the top of the brain near the back. The parietal cortex is a center of higher-level cognition that processes the planning of movements, rather than the details of how movements are executed.
An implant in this area, allow the goal of an action to be conveyed directly to the robotic limb, producing more natural fluid motions as well as reducing the number of neural signals needed to control its movement.
The implants differ from those in Braingate, which placed electrodes in the motor cortex, the part of the brain directs voluntary physical activity. Since motor cortex directly controls many different muscles, so for any one gesture, patients had to painstakingly focus on which muscles to activate for each specific component of the gesture. With these implants the patients could still control a robotic limb, however the movement was delayed and jerky.
Entertainment and gaming
Entertainment and gaming applications have opened the market for nonmedical brain computer interfaces. Various games are presented like in where helicopters are made to fly to any point in either a 2D or 3D virtual world. BrainArena, allows the players to play a collaborative or competitive football game by means of two BCIs. They can score goals by imagining left or right hand movements.
Emotiv EPOC allows one to control keyboard and mouse of your laptop as well as move characters in games. MUSE is to allow you to control your iPhone or Android device with your mind power. With ThynkWare, anyone can use their thoughts to control their smartphones, tablets, home, office, tv, robots, and even clothing.
Mind-controlled telepresence robot
A relatively new field of research is Telepresence that allows a human operator to have an at-a-distance presence in a remote environment via a brain-actuated robot.
A telepresence robot developed at the École Polytechnique Fédérale de Lausanne (EPFL) that can be controlled by thought may give people with severe motor disabilities a greater level of independence. Successfully put through its paces by 19 people scattered around Central Europe – nine of whom are quadriplegic and all of whom were hooked up to a brain-machine interface – the robot handled obstacle detection and avoidance on its own while the person controlling it gave general navigation instructions.
The new report, “Neuroscience, Conflict and Security”, formed part of a series that examined the impact of neuroscience on society, dealing specifically with the potential application of advances in neuroscience to the armed forces and security personnel.
A key advance in neuroscience has been improvements in real-time neuro-imaging, which can indicate in great detail which parts of the brain “light up” when undertaking certain activities. One of its applications could be to screen potential recruits for a specific role, for example to see if they are temperamentally suited to be a commander, pilot or diver.
“At the moment it’s very much a case of taking people on and subjecting them to high-stress exercises and choosing the ones who make it,” says Flower. “If they could be subjected to imaging during assessment you could identify who has good risk-taking behaviour, strategy and planning ability, or 3D analytical skills.”
Brain scanning could also speed up and improve target recognition or identify changes in surveillance satellite images by recognising subconscious objective identification rather than an operator having to process and actively react.
“It has been discovered that when you show the brain different images, it spots the differences between them even though they may not reach conscious awareness,” says Flower. “Wearing a helmet like a hairnet can pick up a spike in brain activity which you can correlate to differences identified between two images, even if they were flashed up too quickly to process consciously.”
That potentially has the ability not only to speed up the process of target selection but also improve accuracy. It could also reduce problems associated with fatigue, which is a big issue facing people whose job involves scanning images for a long time, especially in the dark, such as surveillance UAV operators.
The obvious application for the military is mind-controlled weaponry and remotely-piloted aircraft, which could make operation and reactions far faster. “If you couple that with your subconscious mind being much faster at dealing with information you can see a situation sometime in the future where you’re not thinking about flying the aircraft, but your subconscious is doing it without interfering in any way,” says Flower. “You would probably have a much better appreciation of an incoming threat and fire off a couple of missiles without having to consciously think.”
Like automated weaponry and battlefield robotics, however, these new techniques could require an overhaul of ethical guidelines, especially with regards to civilian casualties. Currently the last person who gave the order to fire is responsible, but if it came from the operator’s subconscious, the line becomes blurred.
Flying manned Aircrafts and Weaponized UAVs by Mind
The University of Florida recently held an event organizers claimed was the “world’s first brain drone race,” featuring unmanned aerial vehicles powered by the brain activity of contestants. The race was billed as a “competition of one’s cognitive ability and mental endurance requiring competitors to out-focus an opponent in a drone drag race fueled by electrical signals emitted from the brain.
Pilots don electroencephalogram headsets that are calibrated to each wearer’s brain. For example, neuron activity will be recorded when the wearer is told to think about pushing something forward. This activity is then bound to the forward stick on the drone’s controller, so future similar neuron activity will move the drone forward. “Organizers of the event describe BCI as “the utilization of a brain imaging device for the purpose of controlling machines with the human brain and to understand the human’s emotional condition or state.”
University of Minnesota carried out a successful demonstration of a thought-controlled mini-helicopter capable of being piloted through obstacles with around 90% accuracy.
A team of engineers at Technische Universität München in Germany developed an algorithm that can convert brain waves into flight commands. The EEG cables sent electrical signals to a computer, which through mind-control algorithm; could target the pilot’s plane-control thoughts and the computer then converted the electric signals into an action that was carried out wirelessly.
In future Soldiers would be able to control both manned aircraft and weaponized UAVs in all their phases of flight
A researcher from Arizona State University has found a way to control multiple drones using nothing but the power of thought.The controller wears a skull cap which contains hundreds of electrodes that are wired to a computer. The wearer then thinks specific commands, the computer translates them into instructions and the robots obey.
Russian Scientists Develop Mind-Controlled Quadcopter
Zelenograd-based company Neurobotics has designed a Mind-controlled quadcopter, that is able to fly not only to four directions — forward, backwards, right and left — but it can also reach a specific target point, the report said.
“Commands, or ‘conditions’ as we call them, are generated by the sensors on the head of an operator. The person thinks about certain actions at right moments which the system then recognizes and identifies,” Neurobotics director Vladimir Konyshev explained, as cited by the report.
The new technology has a great potential in the future. It would not only help a great deal to the Russian Armed Forces on the battlefield, but its interface could also be used to help people with limited mobility, Konyshev added.
However, the basic limitations of current interfaces are that they allow only decoding of basic commands like left right. The future devices would be capable of capturing and decoding brain signals that are responsible for small, precise movements, to be able to accomplish complex task like landing an aircraft. The accuracy of the system is also required to be enhanced.
Brain Controlled Military robots
Brain-Robot Interaction (BRI) refers to the ability to control a robot system via brain signals and is expected to play an important role in the application of robotic devices in many fields
China Developing mind controlled robot Army
Students at the People’s Liberation Army Information Engineering University in Zhengzhou showed in a recent demonstration that they were able to control the movement of small robots using only their minds. At a demonstration at the People’s Liberation Army Information Engineering University in Zhengzhou, students used the device to send robots trundling in different directions. They were also able to turn the robot’s heads and get them to pick up objects. The Chinese army of the future could see robot soldiers controlled by military commanders’ minds.
The technology faces three major engineering challenges that need to be addressed before soldiers will be seamlessly controlling remote military robots in the battleground. First is need to enhance the efficiency and accuracy of non-invasive BCIs that are slow and somewhat uncertain at present, secondly, they tend to make high cognitive demands on the user, and finally, especially for tele-operation via the internet, variable communication delays are a significant problem.
Until then, most of the systems being developed currently are adopting a ‘shared-control’ approach, equipping the robot agent with a degree of intelligence to allow it to work semi-autonomously.
BNCI Horizon 2020 project
European Commission has released a roadmap on BCI: “BNCI Horizon 2020 project”, with the objective of providing a global perspective on the BCI field now and in the future. Many of the applications are centered on the needs of the disabled community for serious injuries. BCIs will provide people with more awareness of their own biological and mental state. BCIs will also promote amelioration of lost function.
Hence, interfacing with the brain directly will be on the forefront of both societal and medical evolution. Relevant application areas include social interaction and recreation, occupational safety, quality of life, independent living in old age, and (occupational) rehabilitation.
In the year 2025, there is expected to be a broad range of brain-controlled applications which, according to the BCI roadmap, will be standard in medical treatment and therapy and also in monitoring personal health.
BCI technology has been advancing at the rapid phase, so that it has now become possible to externally control computers, smartphones, or even vehicles, with thought.
References and Resources also include: