Mind games: why NeuroGaming is the future

How far are we from fully controlling games with our mind? VG247's Dave Cook speaks with neuro-technology expert Zack Lynch to discuss why sensory tools like Oculus and Google Glass are the future of gaming.

Back in 1991 John Carmack and John Romero formed id Software in the hope of one day turning the holodeck concept from Star Trek into a reality. As young, renegade coders that was their endgame, but here we are years later still working towards that dream.

It's getting closer by the day.

A few weeks ago Zack Lynch, managing director at leading neuro-technology firm NeuroInsights orchestrated the world's first NeuroGaming conference in San Francisco, which brought together leading names in sensory gaming, therapy and innovation under one roof.

Together academics discussed the evolution of control within the games industry and how when combined disparate sensory control methods including – but not exclusive to – emotion tracking, sweat monitoring, eye and head tracking, motion control, smell receptors and VR headsets will usher in a sea change across the board.

I recently spoke with Lynch about the state of innovation in console gaming today, and how studies in sensory control will spark the next big thing for the industry. Having published a seminal book called The Neuro Revolution: How Brain Science is Changing Our World, he has seen the field of neuro-software grow rapidly over the years. He told me the sector is on the cusp of realising greatness in the way we design and control games.

Devices like Kinect have shown that home sensor tech needn't cost the Earth.

“How I'd define 'NeuroGaming' is – and this is an evolution in process – but it's really where the mind and body meet gameplay”, Lynch explained. “It's about leveraging principles of neuro-science, but it's also about taking advantage of all the new input and output technologies that are becoming available because of low-cost sensors.

“So on the input side we have a whole slew of new technologies that can drive gameplay in fundamentally new ways, and up the capacity for people to play using new game mechanics. Those include brain wave data, pupil dilation, hand and body gestures, facial expression analysis, as well emotional and cognitive state.

“All of those will drive and analyse how games are designed, and also drive gameplay. So that's on the input side, but on the output side we've got a whole range of new technologies emerging like augmented reality in the form of Google Glass and other companies, virtual reality from Oculus, scent activators that are emerging, to tie more of our senses into the game and haptic technology.”

All of these emergent technologies may seem to detract from the old notion of sitting down to play games with a physical controller, but the margin for innovation – specifically in console technology – is arguably shrinking.

I wrote a piece recently about the danger of PS4 and Xbox 720 games delivering more of the same experiences as current rigs, but with enhanced visuals and dynamic effects thanks to an increase in processing power. This – to me - isn't really innovation in the broad sense of the word, and what we really need is to reappraise the idea of play itself.

PS4 is certainly full of new ideas, but are they truly innovative?

Lynch shared his thoughts on this particular danger, and I asked if the emergence of a true sensory platform could pose a real danger to Sony, Microsoft and Nintendo if they continue to simply update their hardware without reinventing the wheel. On the other hand, reinvention is fraught with risk, so it's likely we'll only see such a format emerge from indie teams with much less to lose.

“These consoles were imagined a decade ago and then reimagined many times,” Lynch replied. “Even the latest consoles that are coming out; while they have the latest chip-sets and phenomenal, world-class graphics and engaging storylines – I mean, the stuff that I see always amazes me – but, there's no way that any of these companies could take into consideration technologies that are just coming out right now.

“They had to start building these consoles four years ago based on technology trajectories, manufacturing capacities and getting game producers to build software. My vision is that you may see a parallel development – a convergence of cheap sensors – with an open API.

“I think some company is going to come along and figure out how to merge these different devices into a consistent, interactive experience – a plug and play, if you will – such that what we see are indie game developers who aren't making money on mobile, switch over to this type of platform. They'll play with it and make a living.

“So I think what you'll see over the next two-to-three years is the emergence of an independent game platform community evolving, which will – not necessarily give the game consoles a run for their money yet – but in time the new game mechanics and new things that will be possible will really drive whatever this future platform might be”.

Is Valve preparing to unveil a platform that brings together several sensory mechanics?

Lynch believes that many companies are already taking steps towards realising a gameplay experience that functions on a range of sensory levels. You can see the signs all around you today, such as Valve's tinkering with VR devices and wearable hardware, and other examples. The problem – Lynch feels – is that “Nothing has caught fire yet”, but whoever does ignite first may be met with failure for daring to innovate.

“It's definitely risky,” Lynch stressed. “Many will fail. Look at how many mobile games fail each day. I can tell you that this IS going to happen, I know this area is going to take off, it makes too much sense from too many directions, and all the drivers are in place.

“Who's going to win? Where's that group going to come from? Is it going to be a group of 30 guys from Bombay, who sit down and get a hold of this tech and pop out with a $199 gaming device in three years, with some new content? Is it going to be a team from Belarus? I don't know. It's a big world out there and everyone has access to the technology now.

“I think Google Glass is going to help act as a gateway drug for people to begin wearing technology on their heads. It's use of miniaturisation technology will allow new sensors to be placed on heads, and as that whole system gets smaller and smaller, that will act as a pretty detailed technology to watch and see how it emerges”.

We're not that far into our discussion and already my mind is racing trying to picture what such a sensory gaming console – if it even ends up being a console – will look like, and how it might feel to play. So used are we to the notion of taking controllers, mouse and keyboard in hand that I don't think any of us can accurately comprehend it yet.

The open platform nature of Oculus will result in much innovation in the indie circuit.

I asked Lynch to try and paint me a picture of what it will feel like to essentially jack our nervous system into the gameplay experience through these many facets of neuro-technology, and posed him the big question – 'How far are we from being able to control games with our mind?' This might sound ludicrous, but companies are working towards this goal as you read this piece.

“I think the problem with that question is that it assumes games will be driven by your mind, “Lynch clarified. “NeuroGaming is not just about having brain control over games. It's about having your entire nervous system connected to the gaming experience, bringing together all of your senses as drivers and outputs through a gaming system.

“You're not going to have a game driven by EEG (a method of recording brain activity), by a neuro-sensing device alone. You're going to have that plus focus, eye-tracking software, facial analysis, heart rate monitors and sweat. You're going to have all of these combining together to give the game designers new information from which to drive and develop fundamentally new gaming experiences.”

What's interesting – and slightly nerve-racking – is that Lynch believes that sensory technology is already the focus of an emerging arms race in many walks of life. Much like companies sprinted to stake a claim in cyberspace as part of the 'Dot Com' boom of the early '90s, he views the battle for our minds to be the next logical step in the world of technological research. He called it the 'Knowledge Economy', and it includes gaming.

Google Glass could spark real interest in commercially-available sensory experiences.

“There's already a gold rush,” he explained. “The brain is the resource of the future. It's the source of the knowledge economy, the source of creativity. Any way that individuals and companies can figure out how to improve the mind, and make it more productive – that's just he story of human history.

“We spent thousands of years developing tools to improve our capacity to grow and distribute food, and then hundreds of years to develop goods and services to distribute them worldwide efficiently. We've spent the past 50 years working on how to distribute and make information extraordinary cheap across the planet, and that has ramifications for both the previous revolutions before it, and then the neuro-revolution will super accelerate all of those.

“You can already see it. Some of the technology is there, some of it is not. Where I'm feeling like it's going to start taking off is in the gaming space from a non-regulated perspective. It's going to make some absolutely fun experiences possible.

“I mean, we're talking about new game mechanics, new engagement mechanics, highly immersive experiences, and experiences that aren't highly immersive but that will fundamentally provide new ways of entertaining billions of people worldwide – and that's a good thing”.

Although Lynch believes it to be a good thing, he realises that – as with all emergent technology – there will be an initial stigma attached to the idea of plugging ourselves into game experiences so intimately.

Tablets had their initial sceptics, but now they are a regular feature in many homes.

It's intrinsic to the very concept of innovation as a whole, but just like we warmed to the ideas of touch, motion and even the Internet itself, Lynch believes NeuroGaming will become accepted into the fold over time.

Lynch concluded, “Any technology ever developed by humankind has both promise and peril. It's a double-edged sword. There are interesting ethical implications as these technologies advance, and we need to keep that in mind as they advance.

“At the same time the promise, the pay-off, the value that can be created by these technologies is hopefully much greater than the downsides. It didn't stop us from electrifying the entire planet with electricity, or developing intercontinental ballistic missiles.

“So neuro-technologyisn't going to get stopped, and I think the pay-off for health, education and all sectors of the economy are interesting. If we can do intelligent, accelerated learning technologies – that will have a profound implication for the nature of competitive advantage – not just for individuals, but for firms, nation states, and that's why I coined the term 'Neuro-Competitive Advantage'.

“I believe neuro-technology is the next form of competitive advantage”.

What's your view on the prospect of NeuroGaming, and the idea of games that utilise every facet of our nervous system to drive the notion of gameplay to new levels? Is it fraught with potential danger – addiction, the blurring of reality or fantasy – or is it the next logical step for the games industry? Let us know below.

We'll have more discussions with other leaders in the NeuroGaming field on VG247 soon. Stay tuned.