Operating machines or playing
videogames just with our thoughts is not science fiction anymore. And Anton Nijholt (see photo on the left), full professor of computer science at the University of Twente, is one of the
researchers working at building so called brain-computer interfaces for these
purposes.
At CHI 2008, he is one of the organizers of the
workshop on Brain-Computer
Interfaces for HCI and Games that takes place today. I’ve
discuss with him about these topics and a lot of interesting things came out as
you can read below.

Could you explain what a
"Brain-Computer Interface" (BCI) is and why it is an important field
of research?
“In Brain-Computer Interfacing
(BCI), so-called EEG caps are used to measure brain activity. An EEG cap is a
cap which has electrodes attached to it and these electrodes measure activity
from different regions of the brain. Unfortunately, these electrodes can’t
measure activity other than activity close to the skull. Moreover, due to
user’s movement (e.g., something as simple as winking of the eye), brain signals
can get disturbed. Nevertheless, it is possible to distinguish brain activity
coming from different regions of the brain.

Different regions of the brain
have different functions. There are parts of the brain that become more active
when an anticipated event is perceived. There are also parts of the brain, in
the so-called motor cortex, that are concerned with movements your body and
limbs can make. In fact, moving the tip of your tongue or imagining to move the
tip of your tongue gives already rise to brain activity that can be
distinguished from other activity. There is a mapping from body part movements
(hands, feet, legs, arms, tongue, whatever) to electrical activity in regions
of the brain. Making these movements, observing these movements and, interestingly,
imagining these movements all lead to activity in a particular region of the
brain. When we can measure and distinguish this activity, then we can also
translate measurements to different commands for the computer or to adaptations
of the interface.

BCI is an important field of
research for different reasons. First of all, there are medical applications.
Disabled persons who are not able to move arms or legs can activate particular
brain activity, e.g., imagine a particular movement, to control a prosthetic
device. But there are also professions, e.g., a pilot of a fighter plane, where
the professional can add something to his capabilities by not only using his
hands, feet, and voice to issue commands, but also by activating particular
regions of his brain. This is very useful, but another application area where
we think that BCI can play an interesting role is in games and entertainment.”

And games are indeed one of the
applications your CHI 2008 workshop is investigating. How will gaming be made
more fun by BCIs?
“There are several reasons why the
combination of games and BCI is interesting. First of all, we can think of
entirely new games that are completely controlled by brain signals. For
example, the game Pong, which is a classical virtual tennis game, can be
controlled by brain commands only. There are similar games, e.g. moving an
avatar in a game environment or in Second Life, commanding an avatar to perform
certain actions in the environment, etc. Secondly, probably more importantly,
we can look at commands issued from particular brain activity as an extra to
the already existing ways of controlling game environments. Consider a
situation where we use already our fingers, hands, and maybe voice to control a
game, and this added way of controlling an aspect of the game allows us to win
the game! Clearly, when gamers have access to such an extra facility, they can
improve their scores and their status in the game community. Thirdly, from the
measured brain activity in different regions of the brain we can get
information about the ‘affective state’ of the user. Is the user bored by the
game, is he irritated or maybe angry, is he excited, etc.? Hence, this is about
the experience of the gamer, and knowledge about this affective state allows us
to adapt the game, the interface, and the ways of interaction to what we know
or able to calculate about the users’ affective state. Knowledge about this
affective state allows us to make real-time changes in the game in order to
keep the gamer ‘in the flow’ of the game.

Unlike medical application where
we need to solve a problem, in games we have much more freedom to exploit the
possibilities of brain-computer interfacing. We need to build challenging
environments that put demands on the user’s cognitive capabilities rather than
making it as simple as possible for the user as we want in the case of a
disabled person that uses BCI. In a game, all shortcomings from BCI can be
translated into challenging features.

Some people consider games as the
killer application for BCI. When gamers adopt BCI, then we have a mass market
immediately and we have an audience that is ready to explore new ideas and
insights immediately. While training time generally is considered to be one
problem that often prevents disabled users to make good use of BCI, for gamers
this training time can become part of the game, going from one level to a next
level in the game or beating an opponent.”

Could you give me a few examples of
particularly interesting work that will be presented at your workshop?
“Among others, there will be new
results on how to evoke brain activity in a controlled way by letting things
happen on a display. This allows the user to choose among alternatives. Others
report about experiments where measurement of brain activity is done without
needing a tedious preparation time, using so-called ‘dry electrodes’
technology. Researchers from Microsoft report about their experiments where
they measure engagement and stress of a user in order to adapt the interface
and the interaction to the mental state of the user. Further experiments with
controlling a wheelchair with BCI are presented by researchers from the
University of Graz and their colleagues.”

What sorts of new applications do you
imagine for the future (besides gaming), thanks to the research that is being
discussed at your workshop?
“Medical applications of BCI will
of course remain extremely important. Applications in gaming and entertainment
are very interesting because the application domain is rather about experience,
fun, challenges, excitement, etc., than about efficiency. This allows us to
design game environments where the use of BCI can be exploited in all kinds of
creative ways rather than that having to solve a problem. It is not necessarily
the efficiency, but it is the experience to which BCI should contribute.
However, many other applications are waiting and in a research context many
other applications have already been looked at. As an example, look at the
control of your home environment. There are several types of BCI commands that
can be used in your smart home environment. By letting different things happen
on a screen, leading to different brain activity, the user can choose among
alternatives. In that way, commands can be issued to your home environment
(open the door, turn on the lights, increase temperature, etc.). One may also
look at situations where a user (e.g. a pilot in a fighter plane, a surgeon
that controls his devices) is already using hands and voice to perform his
tasks and using BCI allows him to get more control on the devices and the
environment.”

How
far away are the applications of BCI?
“Medical
applications are already there. There are patients that use BCI to type e-mail,
to control a disease or to control a prosthetic device. But then we speak of
almost laboratory situations. Applications on a larger scale will only become
possible when the measurement of brain signals can be done wireless, when no
special preparation for measurements have to be done and when we can reduce the
training time that is needed for controlling some kinds of brain activity. It
depends on the brain signals that are needed for the application, but generally
there are many problems in distinguishing the brain signals that we are after
from other signals that are related to other activities of the user. For
example, sometimes brain activity related to the blinking of the eyes may
disrupt certain patterns that we are trying to measure.”

© 2008, Il Sole 24 Ore. Web report from CHI 2008.