Intended to give paraplegic people back some freedom of movement and thus a better quality of life, so-called brain-computer interfaces (BCIs) measure the person's brain activity and convert the electrical currents into control signals for neuroprostheses. "Controlling by thoughts," as Gernot Müller-Putz puts it in simplified terms. The head of the Institute of Neural Engineering at Graz University of Technology (TU Graz) is an “old hand” of BCI research and is intensively involved with non-invasive BCI systems. He and his team have achieved initial positive results with EEG-based control of neuroprostheses or robotic arms in people with spinal cord injuries over the last ten years. However, until now the control was unnatural and cumbersome because the thought patterns had to be repeatedly imagined. As part of his recently completed ERC Consolidator Grant project "Feel your Reach", Müller-Putz and his team have now achieved a breakthrough in the development of more natural and continuous BCI control systems.
It all comes down to seeing
The TU Graz researchers have succeeded for the first time in controlling a robotic arm purely by thought in real time in the usual non-invasive way using an EEG cap (as can be seehn here in the Video). This was made possible by decoding continuous movement intention from brain signals – something previously impossible. The researchers first examined a variety of movement parameters such as position, speed and distance, and extracted their correlates from the neuronal activity. "The contribution of the eyes is essential here," says Müller-Putz. "It is important that users are allowed to use their eyes to follow the trajectory of the robotic arm." However, eye movements and eye blinks generate their own electrical signals, so-called ocular artefacts in the EEG. "These artefacts distort the EEG signal. They therefore have to be removed in real time. However, it is essential that eye-hand coordination can take place and thus contribute to the decoding of movement requests," Müller-Putz explains. In other words, the visual information helps to capture the intention to move. The unwanted signals of the eye itself, however, have to be filtered out of the electrical activity arithmetically.
BCI detects unwanted movements
It is also essential that one of the BCIs developed by the researchers is able to recognize whether a person wants to start a movement – it can recognize the start of a goal-oriented movement. In addition, another of the research team's BCIs detects and corrects errors, i.e., unwanted movements of the robotic arm; one more piece of the puzzle for a more natural prosthetic control. "The brain's error response can be read off from the EEG. The BCI recognizes that the movement performed does not correspond to the person's intention. It stops the movement of the robotic arm or resets it to the beginning," says Müller-Putz. In the project, the error detection was successfully tested several times in tests with spinal cord injured persons.
People can feel movements of the robotic arm
The TU Graz researchers were also successful with so-called kinaesthetic feedback. "The participants not only see the movements of the prosthesis, they also feel them," says a visibly pleased Müller-Putz. Technically, this was made possible with the help of vibration sensors. These are stuck to the skin on the shoulder blade and track the movements of the robotic arm in finely flowing vibrations. Theoretically, it is also possible for completely paralyzed people to feel movements. "However, we have to consider an application in the area of the neck here," says Müller-Putz, alluding to future goals. First and foremost, the researchers want to improve the decoding of a movement from visual, intentional and kinesthetic information, thereby detecting errors and uniting all four BCI systems in a "quadruple BCI system".
Biography Gernot Müller-Putz
Gernot Müller-Putz, head of the Institute of Neural Engineering at TU Graz, has been involved in BCI research for 23 years. He is the recipient of an ERC Consolidator Grant and has organized the Graz Brain-Computer Interface Conference for many years. In 2017, the German Medical Society for Paraplegia awarded him the Ludwig Guttmann Prize. In May 2018, he was elected to the board of the International Brain-Computer Interface Society, of which he is also a founding member. In May 2019, he received the research award of the State of Styria "Digitalization in Science". He is also a founding member and scientific co-director of the NeuroIS Society.
Additional information on Gernot Müller-Putz's BCI research: TedX-Talk 2019: Making the Paralyzed Move
The project "Feel your Reach" was funded by the European Research Council (ERC) within the framework of an ERC Consolidator Grant for Gernot Müller-Putz to the amount of around 2 million euros. The research area is anchored in the Field of Expertise Human & Biotechnology, one of five strategic focal areas of TU Graz.