Brain-Computer Interfaces: The video game controllers of the future

With virtual reality now looking distinctly normal, brain-computer interfaces are look set to become the futuristic tech on the gaming horizon. We discover where the technology is at now, and how it could transform the way we play in the future

When it comes to brain-computer interfaces (BCI) and their use in video games, it can be hard to separate fiction from reality. Valve legend Gabe Newell has confirmed he is researching the technology, and in the latest series of Black Mirror Charlie Brooker painted a terrifying portrayal of how BCI tech could develop. While it may seem far-fetched, however, here and now the technology is already proving its worth.

While not yet really an option for consumer gaming, BCI games are already being used for a host of different health-related projects, creating a whole new way of thinking about how we treat a variety of conditions.

But as time marches on, BCI could have a transformative impact on the world of video games.

“This technology has really commoditised recently. Before that, brain imaging wasn’t realistic unless you were willing to spend many thousands or even millions of dollars,” explains Chris Foster, a researcher at the University of Victoria, Canada. “Today we have devices like the OpenBCI, Emotiv, and the Muse which are affordable for both developers and consumers. That makes the idea of using it for a video game much more realistic.”

The healthy option

When it comes to applications for both invasive and non-invasive brain-computer interfaces, healthcare currently remains king. But what exactly this entails varies wildly by device.

Image courtesy of the US Army

On the invasive side are technologies such as Synapse, a device developed by Nexeon MedSystems that is implanted in the chest and connected to wires running into the brain. Designed to stimulate precise parts of the brain with electricity when paired with a game, it has already been used to treat conditions such as Parkinson’s.

BCI technology is a fairly common solution to the condition, but Synapse takes things a step further.

“This technology is different from the others because it allows us to record what is called local field potentials: the brain activity,” says Will Rosellini, chairman and CEO of Nexeon. “So we think that we can stimulate to alleviate, but we can also record and get a biomarker for how the device is performing.”

In order to make full use of this potential, the company is developing a software suite that will allow greater disease management for users of the device.

“So gamification of rehab, for example, is something that we’re looking at; can we make taking their medication more fun to drive compliance?” asks Rosellini.

But Synapse is not the only BCI technology that Rosellini is involved with. Through his second company, MicroTransponder, he has developed a vagus nerve stimulator, a technology that stimulates a key nerve in the neck to assist with both physical and behavioural therapies.

And once again, pairing the device with a game experience is vital to its success.

DARPA is hoping to extinguish those memories faster by giving soldiers a vagus nerve stimulator and having them play the video game Bravemind

“We are working with a program where they want to link the stimulation with a virtual reality construct, so Skip Rizzo at USC made a program called Bravemind,” explains Rosellini. “Bravemind is a virtual reality video game where you get Afghanistan, Iraq and Vietnam vets to be immersed in scenes that they control, and by exposing them to the videogame you can complete a delinking of the emotions with the memory, and that has been shown to be important in post-traumatic stress disorder.

“DARPA is hoping to extinguish those memories faster by giving soldiers a vagus nerve stimulator and having them play the video game Bravemind to extinguish their memories faster. So that’s a big, $8 million proposal they started last year.”

In addition, Rosellini says that the technology could be used to help rehabilitate stroke sufferers and relieve addiction to drugs such as heroin. However, the fact remains that the technology is highly invasive, meaning its use is likely to remain limited to conditions that are severely life-altering.

Interfaces without implants

While invasive BCI remains the best solution for some severe conditions, technologies are emerging that combine non-invasive brain computer interfaces with video games for more low-key therapy.

A key example of this is Harvard-incubated BrainCo’s Focus 1, a neurofeedback device that is worn like a headband to improve focus by training certain brainwave frequencies.

“The Focus 1 itself is a headband, it has two electrodes on it on the forehead and one behind the ear. It reads alpha, beta and in some of our iterations also low theta waves,” explains BrainCo game developer Jo Wylie. “It takes them, it runs them through an algorithm based on neurofeedback that we’ve developed and it outputs pretty much a very understandable, passable 0 to 100 scale that we just call the attention level.”

Image and featured image courtesy of BrainCo

There are an array of potential applications for device, which is currently being prepared for clinical trials, but at present BrainCo is focusing on developing it as a therapeutic product for children and teenagers with ADHD. The idea is that the users play games made for the device, which help them to improve their concentration and focus.

One such game that has been developed for the system is Focus Oasis, an Animal Crossing-style mobile game that focuses on providing a fun, positive experience that rewards the player for greater focus.

“You drop into this oasis, this area which only you can access and which has a collection of characters in it. So the idea is you walk around, you explore this nice rich environment and each character you meet has a different request for you. Is one character asks: can you help me do my fishing? Somebody else is like: I’m trying to get all these flowers to bloom, can you help me make all the flowers and the frogs come out?”

The idea behind this, says Wylie, is that the player sees a physical improvement in the world as they focus more; a reward for their improved concentration.

“I really didn’t want to just make more homework for the kids, so I wanted to create something that gives them a sense of this is my space and it’s just for me,” she says. “I’m doing this because it feels good and not because I’ve been sat down with it.”

Getting into games

However, while BrainCo is currently only used as a therapy device, it could also have significant potential as a new form of gaming device.

“In the long run I really, really want to make it a purely entertainment device, which is available to anyone playing any type of game, and BrainCo is slowly going there,” says Wylie.

There is definitely a horror application of this device where as you’re walking around, this device will be able to read when you are most scared

That’s not to say that the technology wouldn’t be applicable to gaming in its current state. While the Focus 1 only touches on the potential of BCI, what it does do, it does well.

“I’d love to do a racing game where you’re just racing each other with how focused you are – that could be a lot of fun!” laughs Wylie. “I could go and make it now. We have an attention level: zero is stopped, 100 is 100 miles an hour, it would be relatively easy to code, but it doesn’t fit into what we kind of want to do with the BrainCo device now at all.”

Nevertheless, there will be chances for other developers to use the technology for these types of games before long. While the device does not yet have a set date for commercial availability, the company is planning to put together an SDK that will allow third-party game developers to create compatible experiences for it. These could in theory take the form of a host of different types of game, but all will provide rewards or responses purely within the gaming experience.

“The training technique, there’s no feedback – it doesn’t buzz your head like some neural feedback devices do, it’s purely through gamification: when you’re in a good place your game rewards you,” says Wylie. “All of the game applications will help the brain, will train the brain, but in the long run we’d love to see the SDK used just as a gaming device, or as a training device.”

Some games, of course, will be better suited to the headband than others. Wylie believes walking simulators and continuous runners are most likely to be well suited to the Focus 1, but there are other genres that could be dramatically improved by the addition of the BCI device, particularly in combination with virtual reality.

“There is definitely a horror application of this device where as you’re walking around, this device will be able to read when you are most scared, so the horror game that sees that when you see spiders you become more scared, so as the game goes on you see more and more spiders, that sort of thing,” she explains.

“So horror games that can learn from you. We’ve been talking about that for a while: once we get an SDK we’re specifically going to be reaching out to horror companies because we think that this could be really, really cool.”

If that sounds a little Black Mirror for your taste, however, the technology does also allow for far more restful gaming experiences.

“Personally I’d love to make this game where you’re in a world, in VR, and just imagine you’re sat on a field and all around you as you concentrate all the flowers open,” she says. “And it’s this immersive experience where you’re literally just sat in a place or stood walking around an area, and you’re controlling it and making it light up, all the colours changing and everything happening as you focus. I think it could be a really amazing artistic image.”

The outer limits

At present, BCI devices – and particularly those that are suitable for consumers – are relatively basic. But in time they are likely to develop into far more sophisticated pieces of technology.

However, exactly how sophisticated this form of non-invasive device could become remains a matter of contention.

We’re trying to predict what are called ‘word vectors’ from an EEG signal

“I don’t think BCI – until we’ve got to a point where we’ve got things in our brains, which is not something that attracts me – we’re not going to get directional BCI where you could think ‘lights’ and the lights come on – not unless you have some pretty, pretty intense, deep-in stuff, “says Wylie.

“Honestly I might be wrong on the directional thing, but from what I’ve seen I don’t think we’re going to be able to pick up words.”

However, Foster is working on a research project that could in time to see something almost of this nature become a reality.

“We’re trying to predict what are called ‘word vectors’ from an EEG signal. The user could think of a noun, such as the word ‘cat’, and we attempt to determine information about that word such as ‘Is it alive?’ or ‘Is it a kitchen item?’ based on the EEG signals,” he says.

“It has been shown this can be done with high-end brain imaging such as fMRI, but these machines are extraordinarily expensive. We’re trying to see if this can be generalized to cheaper commodity EEG hardware.”

Foster says that he will better know whether the concept is likely to work by April, but if it does, it could be hugely impactful for the use of BCI.

“This would allow the collection of far more data and be more explorable for a lower price point,” he says. “This can help us understand how the human brain processes language and in the far future potentially make these sort of brain-computer interfaces more practical and effective.”

Nevertheless, even if non-invasive BCI devices are never able to truly detect words, Wylie believes they could provide a very clear picture of a wearer’s feelings, which in turn could be used to brilliant effect in games.

“I think the peak is going to be in emotional reactions,” she says. “Being able to tell exactly when someone is happy, is sad, is scared, all that type of thing.”

XPRIZE launches contest to build remote-controlled robot avatars

Prize fund XPRIZE and All Nippon Airways are offering $10 million reward to research teas who develop tech that eliminates the need to physically travel. The initial idea is that instead of plane travel, people could use goggles, ear phones and haptic tech to control a humanoid robot and experience different locations.

Source: Tech Crunch

NASA reveals plans for huge spacecraft to blow up asteroids

NASA has revealed plans for a huge nuclear spacecraft capable of shunting or blowing up an asteroid if it was on course to wipe out life on Earth. The agency published details of its Hammer deterrent, which is an eight tonne spaceship capable of deflecting a giant space rock.

Source: The Telegraph

Sierra Leone hosts the world’s first blockchain-powered elections

Sierra Leone recorded votes in its recent election to a blockchain. The tech, anonymously stored votes in an immutable ledger, thereby offering instant access to the election results. “This is the first time a government election is using blockchain technology,” said Leonardo Gammar of Agora, the company behind the technology.

Source: Quartz

AI-powered robot shoots perfect free throws

Japanese news agency Asahi Shimbun has reported on a AI-powered robot that shoots perfect free throws in a game of basketball. The robot was training by repeating shots, up to 12 feet from the hoop, 200,000 times, and its developers said it can hit these close shots with almost perfect accuracy.

Source: Motherboard

Russia accused of engineering cyberattacks by the US

Russia has been accused of engineering a series of cyberattacks that targeted critical infrastructure in America and Europe, which could have sabotaged or shut down power plants. US officials and private security firms claim the attacks are a signal by Russia that it could disrupt the West’s critical facilities.

Google founder Larry Page unveils self-flying air taxi

A firm funded by Google founder Larry Page has unveiled an electric, self-flying air taxi that can travel at up to 180 km/h (110mph). The taxi takes off and lands vertically, and can do 100 km on a single charge. It will eventually be available to customers as a service "similar to an airline or a rideshare".

Source: BBC

World-renowned physicist Stephen Hawking has died at the age of 76. When Hawking was diagnosed with motor neurone disease aged 22, doctors predicted he would live just a few more years. But in the ensuing 54 years he married, kept working and inspired millions of people around the world. In his last few years, Hawking was outspoken of the subject of AI, and Factor got the chance to hear him speak on the subject at Web Summit 2017…

Stephen Hawking was often described as being a vocal critic of AI. Headlines were filled with predictions of doom by from scientist, but the reality was more complex.

Hawking was not convinced that AI was to become the harbinger of the end of humanity, but instead was balanced about its risks and rewards, and at a compelling talk broadcast at Web Summit, he outlined his perspectives and what the tech world can do to ensure the end results are positive.

Stephen Hawking on the potential challenges and opportunities of AI

Beginning with the potential of artificial intelligence, Hawking highlighted the potential level of sophistication that the technology could reach.

“There are many challenges and opportunities facing us at this moment, and I believe that one of the biggest of these is the advent and impact of AI for humanity,” said Hawking in the talk. “As most of you may know, I am on record as saying that I believe there is no real difference between what can be achieved by a biological brain and what can be achieved by a computer.

“Of course, there is unlimited potential for what the human mind can learn and develop. So if my reasoning is correct, it also follows that computers can, in theory, emulate human intelligence and exceed it.”

Moving onto the potential impact, he began with an optimistic tone, identifying the technology as a possible tool for health, the environment and beyond.

“We cannot predict what we might achieve when our own minds are amplified by AI. Perhaps with the tools of this new technological revolution, we will be able to undo some of the damage done to the natural world by the last one: industrialisation,” he said.

“We will aim to finally eradicate disease and poverty; every aspect of our lives will be transformed.”

However, he also acknowledged the negatives of the technology, from warfare to economic destruction.

“In short, success in creating effective AI could be the biggest event in the history of our civilisation, or the worst. We just don’t know. So we cannot know if we will be infinitely helped by AI, or ignored by it and sidelined or conceivably destroyed by it,” he said.

“Unless we learn how to prepare for – and avoid – the potential risks, AI could be the worst event in the history of our civilisation. It brings dangers like powerful autonomous weapons or new ways for the few to oppress the many. It could bring great disruption to our economy.

“Already we have concerns that clever machines will be increasingly capable of undertaking work currently done by humans, and swiftly destroy millions of jobs. AI could develop a will of its own, a will that is in conflict with ours and which could destroy us.

“In short, the rise of powerful AI will be either the best or the worst thing ever to happen to humanity.”

In the vanguard of AI development

In 2014, Hawking and several other scientists and experts called for increased levels of research to be undertaken in the field of AI, which he acknowledged has begun to happen.

“I am very glad that someone was listening to me,” he said.

However, he argued that there is there is much to be done if we are to ensure the technology doesn’t pose a significant threat.

“To control AI and make it work for us and eliminate – as far as possible – its very real dangers, we need to employ best practice and effective management in all areas of its development,” he said. “That goes without saying, of course, that this is what every sector of the economy should incorporate into its ethos and vision, but with artificial intelligence this is vital.”

Addressing a thousands-strong crowd of tech-savvy attendees at the event, he urged them to think beyond the immediate business potential of the technology.

“Perhaps we should all stop for a moment and focus our thinking not only on making AI more capable and successful, but on maximising its societal benefit”

“Everyone here today is in the vanguard of AI development. We are the scientists. We develop an idea. But you are also the influencers: you need to make it work. Perhaps we should all stop for a moment and focus our thinking not only on making AI more capable and successful, but on maximising its societal benefit,” he said. “Our AI systems must do what we want them to do, for the benefit of humanity.”

In particular he raised the importance of working across different fields.

“Interdisciplinary research can be a way forward, ranging from economics and law to computer security, formal methods and, of course, various branches of AI itself,” he said.

“Such considerations motivated the American Association for Artificial Intelligence Presidential Panel on Long-Term AI Futures, which up until recently had focused largely on techniques that are neutral with respect to purpose.”

He also gave the example of calls at the start of 2017 by Members of the European Parliament (MEPs) the introduction of liability rules around AI and robotics.

“MEPs called for more comprehensive robot rules in a new draft report concerning the rules on robotics, and citing the development of AI as one of the most prominent technological trends of our century,” he summarised.

“The report calls for a set of core fundamental values, an urgent regulation on the recent developments to govern the use and creation of robots and AI. [It] acknowledges the possibility that within the space of a few decades, AI could surpass human intellectual capacity and challenge the human-robot relationship.

“Finally, the report calls for the creation of a European agency for robotics and AI that can provide technical, ethical and regulatory expertise. If MEPs vote in favour of legislation, the report will go to the European Commission, which will decide what legislative steps it will take.”

Creating artificial intelligence for the world

No one can say for certain whether AI will truly be a force for positive or negative change, but – despite the headlines – Hawking was positive about the future.

“I am an optimist and I believe that we can create AI for the world that can work in harmony with us. We simply need to be aware of the dangers, identify them, employ the best possible practice and management and prepare for its consequences well in advance,” he said. “Perhaps some of you listening today will already have solutions or answers to the many questions AI poses.”

You all have the potential to push the boundaries of what is accepted or expected, and to think big

However, he stressed that everyone has a part to play in ensuring AI is ultimately a benefit to humanity.

“We all have a role to play in making sure that we, and the next generation, have not just the opportunity but the determination to engage fully with the study of science at an early level, so that we can go on to fulfill our potential and create a better world for the whole human race,” he said.

“We need to take learning beyond a theoretical discussion of how AI should be, and take action to make sure we plan for how it can be. You all have the potential to push the boundaries of what is accepted or expected, and to think big.

“We stand on the threshold of a brave new world. It is an exciting – if precarious – place to be and you are the pioneers. I wish you well.”