Mel Slater's Presence Blog

Even Though You Know it is an Illusion...

2011-12-19T18:30:00.014+01:00

Today was Andrea Brogni's PhD exam at Universitat Politècnica de Catalunya. He started his PhD with me some time in 2006 or 2007. His thesis is entitled Haptic Illusion in Virtual Environments. The idea is that instead of using a haptic interface to make the experience in a virtual environment a physical one, we use the brain. First he showed that if you let people 'touch' objects in VE without using a haptic interface, then they will experience sharp objects as sharp and smooth objects as smooth. Some evidence suggested that people have a physiological response to sharp objects that they do not have to smooth objects. Moreover, the experimental design ruled out the possibility that this was purely suggested by vision. Next he showed that it is possible to give the illusion of physicality when moving objects by forcing people to use their muscles. In other words if you try to move a 'heavy' object, nothing happens unless you exert sufficient muscular effort (as measured by EMG). This is an idea I've had floating around since the early 1990s, and indeed Martin Usoh and I wrote an EPSRC grant to further this, but it was rejected. I'm happy to see that finally it was realized and investigated by Andrea and done very well. We will carry on with this in the future. As he says though, it is not an attempt to 'replace' haptic interfaces, but an alternative method to experience physicality in virtual environments.

Actually why I'm writing today is because Andrea said something that reminded me that there is an issue I've had on my mind to write about for some time. When I talk about 'presence' (place illusion or plausibility) I always put a rider 'in spite of knowing that this is an illusion'. I.e., place illusion is 'the illusion or sensation of being in the virtual place in spite of the sure knowledge that you are not there' or plausibility is 'the illusion or sensation that what is happening is real, even though you know that there is nothing real happening'. Why 'even though you know that it is an illusion'? This was pointed out to me by someone as being redundant, or even wrong. But I had the feeling that it was a necessary part of the definition. This is why ….

It is because the feeling that is associated with the knowledge that the virtual experience is an illusion is part of the overall sensation associated with the illusion (e.g. of PI or Psi). Let's imagine that someone did not have knowledge that it was an illusion. Then for them there is no question of 'place illusion' since as far as they are concerned that are in that place, just as they would be in any place in everyday life. Hence there is no feeling associated with an illusion, since subjectively there is no illusion. Now it is quite a different issue to engineer a situation where participants do not know that it is an illusion. This is equivalent to subjectively 'reproducing reality' for that person. From a scientific and engineering point of view this is a more profound objective, and currently unachievable. Even in the famed 'holodeck' of Startrek people know that what they are experiencing is an illusion, unless it were possible to engineer in them the 'forgetting' of the fact that they entered from the real world into the holodeck, and the suppression of their knowledge about how the holodeck works. So in a sense the more modest goal is to get people acting in a VE in a realistic manner *even though they know that it isn´t real* which is different from the goal of getting them to falsely 'know' that it is real (and therefore the question of trying to get them to act realistically wouldn't arise, since by definition if they didn't know it wasn't real they would act realistically). At one level this seems like 'hair splitting' and only philosophical, but I think it actually reflects different engineering goals, as well as scientific ones. An analogy would be that to remove that part of the definition is equivalent to saying that we could make virtual reality operate like a dream. While the dream is happening you do not know that it isn't real - it is only afterwards when you wake up that you realise this. I think that at the moment this objective isn't realisable - except perhaps by the use of drugs, hypnotism, or neural implants.

Presence at SIGGRAPH 2010

2010-08-02T12:47:00.002+02:00

One of the problems with ‘presence’ has always been that of measurement – since ‘place illusion’ (the sensation of being in the place depicted by the VE displays) and ‘plausibility’ (the sensation that what is happening is really happening) are both qualia, they are feelings that cannot be directly measured.

Some years ago I put forward an analogy with colour science - in the now defunct online journal Presence Connect – see [1].

For example, the sensation of seeing the colour ‘red’ is also a qualia, much beloved of scientists and philosophers who study consciousness. In colour science there is a physical function, the wavelength distribution, that describes the energy distribution of light emitted or reflected by a surface patch. Yet what we see is not simply some simple function of this physical energy distribution but a complex, and not completely understood interplay between the physics of colour and our perceptual systems. Moving on from there to the sensation of ‘red’ and our consciousness of seeing red is another far leap into the science of consciousness.

With respect to ‘presence’ the physical basis is the type of immersive system used, its properties and capabilities. This physical basis then becomes transformed into our perception and action within an alternate reality. Again how this transformation occurs is a problem for significant study.

There is a successful quantitative and predictive theory of colour science, that shows how an ‘average observer’ is likely to respond to patches that emit light with specific energy distributions. This success is partly built on the psychophysics of colour matching experiments. We applied an analogy of this idea to ‘presence’ (Place Illusion and Plausibility) and carried out an experiment to show how this idea could work. This has just been published at SIGGRAPH 2010 [2].

This shows how to create functions that predict how the ‘average participant’ would respond to particular system configurations in terms of presence. The paper provides a methodological counterpart to the earlier theoretical paper [3].

There is an associated youtube video, and a version of the presented slides is given below.

1. Slater, M., A note on presence terminology. Presence connect, 2003. 3(3).

2. Slater, M., B. Spanlang, and D. Corominas, Simulating Virtual Environments within Virtual Environments as the Basis for a Psychophysics of Presence. ACM Transactions on Graphics (SIGGRAPH) (TOG), 2010. 29(3): p. Paper: 92.

3. Slater, M., Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments. Philos Trans R Soc Lond, 2009. 364(1535 ): p. 3549-3557.

2010-07-04T13:41:00.003+02:00

Presence in a Another Body

We recently published a paper on how we used virtual reality to give people the illusion that their body was temporarily that of another (First Person Experience of Body Transfer in Virtual Reality). In particular we were able to show that to some extent some men can be given the (temporary!) illusion that their body is that of a girl.

The image shows an overview of the scenario. The experimental participants were located in or near the seated girl's body, via wearing a head-tracked wide field-of-view head-mounted display. In one condition of the experiment, when they looked down towards their body they would see the girl's body instead, as if their eyes were located in the same position as hers. If they turned to look to their left, they would see the body of the girl in the virtual mirror, with its head movements matching their own. When the standing woman stroked the arm of the seated girl, they would feel this on their own arm.

This paper seems to have generated a lot of interest judging from press reports and the PLoS ONE metrics.

Some of the comments about the paper are quite funny, and it seems to have awakened kinky fantasies in some men. Other comments are off the mark believing (spurred on by some journalistic takes on the article) that we were transforming men into women. Indeed one angry comment on the paper notes that to give men the experience of being female we would have to also simulate all of the social conditions that women are subject to, and also physiological processes such as menstruation, etc..

I agree very much with this. We were not giving men the experience of what it is like to be a woman! We were generating an illusion that is very hard to describe or understand unless you have experienced it. It is an illusion that the different body that you see when you look down at ‘yourself’ in virtual reality, or when you see ‘yourself’ in a virtual mirror, is somehow your body (even though you know it isn’t).

The brain seems to be quite liberal in deciding what is part of your body, and such illusions have shown that it is not difficult at all to trick the brain into believing that something is part of your body when it is not. It is important to realise that this ‘trick’ does not happen at the cognitive level, that is you never believe that the fake body or body part is really part of you. Rather it is at some lower perceptual and proprioceptive level that you don’t have much conscious control over that this happens. So it is like ‘presence’ in virtual reality - you know for sure that this is not your body, but nevertheless it feels like it is.

These illusions stem from an initial insight by Botvinick and Cohen known as the ‘rubber hand illusion’, and see also the New Scientist video with Olaf Blanke. Here people experience the illusion that a rubber hand is their hand. However, no one would then go on to say that people who experience the rubber hand illusion know what it is like to have a rubber hand! Similarly we do not claim that a male who has the illusion that his body looks like a female one knows what it is like to have female body. These are different things at quite different logical levels.

I hope you find the article interesting, and on the PLoS ONE web page you can see a video which gives some idea of how it looks to the experimental participant. However, the only way to really know how the illusion feels would be to experience it. Welcome to Barcelona.

The Illusion of Violence

2010-01-24T23:57:00.005+01:00

If you see some people fighting in the street, how do you respond? Some people may intervene and try to stop the fight, others may try to get help, others may just watch, join in one side or the other, or do nothing but get away as quickly as possible. How people respond to violence is an important topic in social psychology, and the issue goes back to a case that happened in the US in 1964, the violent murder of a young girl while apparently 38 bystanders stood by and did nothing. There is controversy over what actually happened, about whether there really were these bystanders, but anyway a whole area of research was opened up in social psychology to try to understand this ‘bystander effect’. One of the problems in studying this is that, of course, it is impossible to carry out experimental studies of how people respond to violence that they might encounter by chance in a public place. The social psychologist Dr Mark Levine of the University of Lancaster has come up with some ingenious ways to try to study this problem, and actually his view of how crowds behave is not quite so negative as the popular view that crowds are bad and that they encourage or don’t prevent violence. A few years ago Mark and I talked about the possibility of using virtual reality to study this issue, since in VR one can set up apparently contingent violent confrontations between virtual people and then study how real people respond to this.

We know from research into presence that people do, in VR, tend to respond as if situations and events were real. (On this topic, my paper published in the Proceedings of the Royal Society, mentioned in earlier posts, is freely available online until the end of February).

So if we place people in a virtual reality where there are virtual characters who start arguing and fighting with one another, then this might prove an interesting way forward for the experimental study of the topic of the bystander effect.

Actually one reason why we did the virtual reprise of the Stanley Milgram obedience experiment was that we needed to convince sceptical EPSRC project reviewers that it is the case that people do tend to respond realistically. (On this point too another paper has been published recently using brain imaging to try to understand what happens when people do experience that Milgram paradigm). The EPSRC were convinced, and Mark Levine, Prof. Jian J Zhang and I obtained project funding, with this as the major application.

We carried out a first pilot study on the issue of people's responses to violence in virtual reality which has now been published. Actually this did not portray violence but aconfrontation between two football fans that would eventually lead up to violence. The results were very encouraging, and we are now carrying out a full study.

Other News

On the topic of ‘response as if real’ there are some other recently published papers from our group that might be of interest. The first examines whether illumination realism makes any difference when people experience standing over a virtual pit. Does it help to have dynamic shadows and reflections? A pre-publication version is available online.

The second shows that when people walk across a narrow beam in virtual reality they tense their back muscles to avoid falling. This is in press with IEEE Transactions on Visualization and Computer Graphics, and is available as a pre-publication version.

The third shows that how people scan their eyes over a scene in virtual reality follows a pattern that is similar to what might occur in reality.

The fourth shows that people exhibit behaviour consistent with proxemics theory when virtual characters break into their virtual space, which is in press with ACM Transactions on Applied perception

Research Posts Available

I have several jobs available in the group in Barcelona. Take a look at www.event-lab.org over the next few days and you will see the adverts appear.

Illusion is Part of the Definition

2009-08-30T00:58:00.011+02:00

In July there was a conference in Benasque, Spain, Art and Science: Exploring the Limits of Human Perception. We (Mavi Sanchez-Vives and I) teamed up with Nonny de la Peña and Peggy Weil from California, to create a piece based on the idea of immersive journalism. You can read about what it is like to be a prisoner at Guantánamo, but can you get first hand knowledge of what the experience is really like?

We created a scenario in immersive virtual reality, relying on research on body ownership, to give people the illusion that they were in a cell standing in a stress position, while hearing an interrogation going on in the cell next door. There is an article about it in El Pais: Presos de un Guantánamo virtual, and a youtube video: Towards Immersive Journalism: The IPSRESS Experience.

My new paper attempting to give my current understanding of 'presence' has been formally accepted for the Philosophical Transactions of the Royal Society, Series B. This is about deconstructing the concept of presence into two orthogonal concepts: 'place illusion' - the original idea of having a strong illusion of being in the virtual place, and 'plausibility', the illusion that what is happening is really happening. Of course these are illusions, no one really believes that they are in the virtual place or that what is happening is real - the fact that these are illusions is part of the very definition. Also the paper discusses what I think is a good way to understand 'immersion'. Altogether there are four concepts: Place Illusion (PI, based on sensorimotor contingencies), Plausibility (Psi, based ultimately on various correlations between actions and events), immersion (forming a simulation hierarchy) and the virtual body which is the intersection between PI and Psi. I think that it is relatively clear to me now, ideas that I've been trying to find for many years. 'Presence' in the sense of place illusion is not and never really was the main problem - it can be relatively easily achieved most of the time with the right display and especially tracking systems. The real problem is Plausibility. The pre-publication version of the paper can be downloaded from my publication website.

The Ethics of Virtual Milgram at the Royal Society

2009-04-26T17:59:00.020+02:00

Emotions in Man and Machine

On 21st April in London I spoke at a Royal Society Discussion Meeting The Computation of Emotions in Man and Machines. The overall meeting was excellent and if you look at that web page you will see that there will be a webcast archive of all the talks, and also proceedings in a future publication of Philosophical Transactions B. About 300 people were in attendance and the meeting was oversubscribed. I will put a version of my paper online and talk about its contents later. For now I want to concentrate on another issue.

Ethics

During the talk in order illustrate one point I briefly went through the ‘virtual reprise of the Stanley Milgram obedience experiment’ that I talked about in this blog on 28th December 2006 ‘Obedience in Plaça Espanya’.

In the discussion after the talk there were two people in the audience who raised the issue of ethics. I know that mentioning Stanley Milgram’s obedience experiment is like raising a red rag to a bull for some psychologists, and I agree that the original experiments in the 1960s were problematic. However, I also urge people interested in this issue to read Stanley Milgram’s own forceful discussion of this issue in his book: Obedience to Authority: An Experimental View.

Also the argument rages today about the ethics of the original experiment. See for example, ‘Milgram, Method and Morality’ by Charles R. Pigden and Grant R. Gillet in the Journal of Applied Philosophy 13 (3) 233 – 250, which is a response to a recent partial replication of the obedience experiment by Jerry M. Burger.

Readers should check the British Psychological Association’s Code of Ethics and Conduct, March 2006 ‘3.3 Standard of Protection of Research Participants’, and also the information sheet and consent form given to our participants is available as are the answers by some participants to a letter that was sent 6 months after the actual experiment.

The Power of Virtual Reality

First, why did we do the experiment? The answer actually was not to probe Milgram’s original question of obedience to authority. In the original 1960s experiment the experimenter deliberately used Authority of a Professor at a prestigious institution to attempt to persuade participants to carry out actions that would have normally been against their own moral principles (causing harm to a stranger, and continuing to do so in spite of that stranger’s strong protestations to the contrary). In our virtual reprise the experimenter did not at any time attempt to persuade the participants to continue against their own inclinations, in fact they were told in writing and verbally several times before the study began that they could withdraw at any time without giving reasons. What we were interested in was whether causing ‘harm’ to an entirely virtual character who protested about the shocks ‘she’ was getting would cause people anxiety so that they would want to stop. In other words in spite of knowing for sure that nothing ‘real’ was happening, would people still find the experience unpleasant, and would they still want to stop even knowing that it was virtual? Moreover, if they found it unpleasant and yet did not stop, why would they continue?

Why are these questions interesting? The fundamental answer is that we want to explore the power of virtual reality to simulate situations in reality and cause responses in people that are similar to those of real life. By ‘responses’ here we mean mainly those automatic responses that occur in spite of the person’s full knowledge that the situation is not real. For example, in an earlier study we had put people in front of audiences of virtual characters, who behaved either very negatively towards them or very positively – and the interesting thing is that although everyone knew that there was no audience there, they still responded with anxiety to the negative audience, and with a kind of joy to the positive audience .

Now Milgram’s original question remains: why is it that people can be persuaded to carry out atrocious acts at the behest of authority, acts that are against their own moral principles? We see examples of this every day in the news. This issue is something that is really worth studying, something that is as urgent today as it was in the 1960s, in the 1930s and 1940s, and probably any previous time in history. However, it is very difficult to study – the ethical concerns raised by Milgram’s original experiment stand: we cannot allow people to believe that they really are causing harm to another person in order to see how they react. On the other hand having people watch videos and asking how they would react, or even having them imagine the situation and their likely responses simply isn’t sufficient for scientific study – no one knows how they would react in such situations.

Realistic Responses

I believe that our research has shown many times that in virtual reality, and under the right conditions, that people do tend to respond realistically to what they experience (actually this was the main subject of my talk at the Royal Society). However, their knowledge that what is happening is not real tends to dampen down their responses. So their base level responses (physiological responses, feelings, emotions, automatic thoughts) are genuine, but ultimately they can use their knowledge of the situation to control their overt behaviour. For example, in the virtual reprise, when people were asked why they continued in spite of wanting to withdraw, they would invariably say something like ‘… because I kept reminding myself that it wasn’t real’. In any event I believe that what we have shown in this work is that virtual reality can be used effectively to study how people respond in extreme situations, how, for example, the terrible events with which we are only too familiar can be caused by people who in ordinary circumstances would be horrified about such things.

Responsible

I do not accept the argument that causing some stress to participants in an experiment is not ethical. These are adults, who freely agree to participate in the study, and who are told that they are free to withdraw at any time, and even warned that they may experience stress. If they decide to continue in spite of experiencing stress that is their choice, they are under no obligation to continue. People voluntarily choose to engage in activities that are far more stressful than anything we have ever subjected them to in virtual reality – watching horror movies, doing dangerous sports, even simply attending a football match might be a highly stressful activity. Don’t forget – these are adults who are responsible for their own actions, and provided that they are not tricked or deceived into entering a situation that might cause them difficulties without forewarning, it is up to them to participate or not. Of course there are limits, and a major ethical consideration is to weigh up the benefits of the research in terms of knowledge gained balanced against any negative aspects of the experiment.

Desensitisation

The other issue is ‘desensitisation’ – by participating in this experiment could it make participants more likely to actually carry out cruel acts in real life? This was suggested by one of the audience members at the Royal Society talk. Actually the question is an empirical one – does involvement in violent virtual scenarios result in greater aggressive behaviour in real life? This is an issue much studied with respect to violent video games, and the jury is still out. See, for example, the paper by Christopher John Ferguson, The Good, he Bad and the Ugly: A Meta-analytic Review of Positive and Negative Effects of Violent Video Games. Of course, the virtual Milgram study presented nothing like the kind of violence one can inflict in video games. On the other hand one could argue equally well that having experienced a virtual reality scenario where you found yourself carrying out an act that causes stress and unpleasant feelings to yourself, that in the future you might not want to do that again, and especially would be forewarned about somehow getting trapped to do this sort of thing in reality. Hence such an experience might open the door to self reflection and minimise the chance for later aggressive behaviour. But, as I said, this is an empirical question, not one that can be settled by argument or simple introspection.

Indignation
Therefore, the following clause from the BSA’s code of ethics comes into force:

‘Obtain the considered and non-subjective approval of independent advisors whenever concluding that harm, unusual discomfort, or other negative consequences may follow from research, and obtain supplemental informed consent from research participants specific to such issues.’

Our complete experimental design was submitted to our University’s Research Ethics Committee, and was discussed as a full application (i.e., discussed by the Committee and not subject to Chair’s action). It was deemed an appropriate experiment. I would suggest that the person in the Royal Society audience who claimed in public that the ethics committee was wrong, who had the temerity to claim this after listening to my five minute discussion of the experiment, without knowing anything whatsoever about its details, nor about the deliberations of the committee, and presumably without ever having read the paper – that this was an example of ‘indignation’ that has no place in a scientific meeting – least of all in a place like the Royal Society.

Transcending Reality (with a diversion on Statistics)

2009-04-05T20:38:00.011+02:00

My Senior ERC grant called TRAVERSE started 1st April 2009. TRAVERSE stands for 'Transcending Reality – Activating Virtual Environment Responses through Sensory Enrichment'.

What is it about?

I use the term ‘transcending reality’ (TR) in two ways as a noun phrase and a verb phrase. A ‘transcending reality’ is one that replaces physical reality by a virtual reality, such that you respond to the virtual reality as if it were real. However, to ‘transcend reality’ is to go beyond the boundaries of physical constraints, when the virtual reality gives you the strong illusion that you've gone beyond the boundaries of physical reality. In these ‘non realistic’ applications of virtual reality I nevertheless expect people to respond to it as a TR. The overriding background objective of this research is: to maximise the probability that participants will act as if the immersive virtual reality were real (TR).

The technical research includes the main components of virtual reality - computer graphics and haptics mainly. Haptics hasn't been a strong field for me in the past, but I'm realising its profound importance - and learning more about it (see 'Haptic Rendering' edited by Ming Lin and Miguel Otaduy). However, the research is mainly interdisciplinary, including computer science and neuroscience.

There's another important aspect - there will be several experimental studies, and of course statistical analyses of the results. For a long time I've known that the classical approach to statistics - significance levels, type I and II errors, power, Neyman-Pearson Lemma, etc - is fine, but ... it doesn't make sense. What is a 'significance level'? It is the probability of 'rejecting your null hypothesis conditional on the null hypothesis being true' P(reject H0 | H0). Who cares?

What we're really interested is the probability of the hypothesis given the observed data: P(H | O), where O stands the the observations. This isn't allowed in conventional statistics since making probability statements about a hypothesis doesn't make sense - since the probability of an event has the interpretation that it is the ratio of occurences of the event to the number of times that the event could have occurred in a long run series of independent and identical trials. Clearly the truth of a hypothesis cannot be an outcome of an experimental trial - from this point of view P(H) = 1 (it is true) or P(H) = 0 (it is false), but we don't know which one of these holds.

This way of thinking leads to Bayesian statistics, where probability is interpreted as subjective degree of belief - so a statements such as P(H) = 0.75 is valid, it means your degree of belief that H is true. From Bayes' Theorem we get P(H | O) ~P(O | H)P(H) (I'm using ~for 'is proportional to'). P(O | H) is often something that can be computed from probability theory, and P(H) is your 'prior probability' for H ('prior' because it is before you get the data). Then Bayes' Theorem allows you to update your probability for H as more and more data is accumulated. In the end two different people who might have started with quite different priors will end up with the same final probabilities (P(H | O)) given sufficient data (O). So I preferred Bayesian statistics, although it does require choosing prior probability distributions.

It is interesting that while the field of statistics has undergone a sort of revolution in the past two decades where Bayesian statistics has become completely acceptable, and considered now part of mainstream statistics, the fields in which statistics are probably used most (in psychology and the social sciences) stick rigidly and ideologically to the sacredness of the 5% significance test.

Let's consider an example of how problematic this is. Suppose this week I do an experiment, and I report results at the 5% significance level. OK. Then next week I do another experiment and I report results at the 5% significance level. And so on for the next 100 weeks. Each of these different experiments I write up in a different paper. They are all accepted (well, of course, this is virtual reality!). So no problem with that. Now in a parallel universe, one also where psychology is dominated by classical statistics, I am very energetic, and I do all 100 experiments in a single week, and I write all the results in one paper and I get exactly the same results (i.e., the same things are 'significant') as in this universe. I then submit the paper for publication, and it is rejected for being statistically unsound! Why? Because ... if you do n tests all at the 5% significance level, then 'by chance alone' on the average 0.05*n of them are going to be 'significant' (think back to the meaning of 'significance level'). Note the only difference in the two universes is that in one I spread the results out over 100 weeks, and put them in 100 different papers, but in this other universe I did them all in a short time period and submitted them in one paper.

How, in the second universe, can we get out of this problem? Well the reviewers of the fictional paper say that I should have applied something called the 'Bonferroni Correction'. What this means, at the simplest level, is that if you do n tests, then you should use a significance level of 0.05/n.

But this is unfair no? If I spread the tests out over many weeks and put each in a different paper, then - no problem. But if I'm especially energetic and do all the tests at once, and then write them all in the same paper, my significance level has to be 0.05/100 = 0.0005. Unfortunately now nothing is significant!

Let's take this argument a bit further. Why pick on me? Why not throw your tests into the pot, and in fact all the tests in this universe. n is infinite, nothing is ever significant, all those fantastic results "it was significant at the 5% level" that we've ever seen are all ... are not supported, statistically invalid, since according to the 'Bonferroni Correction' the significance level is 0, and we can't get smaller than that (at least in this universe).

Now more recently there is another 'new wave' in statistics, based on information theory. I've been reading and learning this recently, and it is ... cool. You don't need prior distributions. You consider the question: what 'information' does this data contain about the possible models under consideration? So, I really like the information theory approach to statistical inference, since it gets to the heart of what the real problem is about, without any mumbo jumbo, weird concepts, strange tricks, and sleights of hand.

If you're interested have a look at http://www.mdl-research.org/. Unfortunately this approach has not reached the mass of practitioners yet, and maybe because there are a lot of new things to learn, with some not so trivial mathematics in the way. However, there is also a really nice practical book that is within this approach: Burnham, K. P., and D. R. Anderson. 2002. Model Selection and Multimodel Inference: A practical Information-Theoretic Approach. 2nd Ed. Springer. Although very practical it also explains the underlying concepts well. For the first time I felt I was doing something really appropriate in statistical analysis using these ideas analysing a recent experiment. However, probably the psychologists will not agree.

Now to get back to the point - for TRAVERSE I'm looking for researchers to fill a number of new research posts at both the post-doc and PhD student level. I expect that applicants will be from the fields of computer science, or cognitive neuroscience with computer science. Knowledge of computer graphics / virtual reality would probably be essential -

- Except for one position - I really want to have a statistician in my group. I would really like to have a statistician who is not orthodox (but who knows the orthodoxy) and is interested in furthering as a research topic, the information approach to statistics, as well as analysing the data of our experiments. Also, I have a strong intuition that the information approach to statistics may also turn out to be an interesting model for the underlying fundamental research questions that we will tackle.

Virtual Pain in Dubrovnik

2008-08-03T17:02:00.010+02:00

I attended the Second PEACH Summer School in Dubrovnik, Croatia 9-11th July, 2008.

The title of the school was ‘Presence: Technologies and Applications’. There were many interesting talks during the week, and I hope that eventually some will be put on the web site, and they are well-worth watching.

The courses were supposed to be more ‘hands on’ this year, with practical applications, exercises and so on. Originally I was going to do an experiment with the students utilising Second Life, in real-time. However, this turned out not to be possible, since there were not appropriate or a sufficient number of computers available.

Nevertheless, in my talk, I wanted to illustrate some ideas in a practical way, rather than just by talking about them.

A central feature of an experience in virtual reality is that you respond to events even though you know that they are not ‘true’ (i.e., the virtual character that you feel compelled to smile back at is not really there, and you know this to be the case). In fact I would take it as part of the definition of ‘presence’ that you know that the illusion is not ‘true’ (but nevertheless finding yourself responding as if it is true). If you didn’t know that it wasn’t true, i.e., if you fully believed that what appeared to be happening was really happening, and that you really were in the place that you appeared to be, then it is no longer ‘virtual reality’ but simply ‘reality’ (for you).

How could I give a demonstration of this, without any actual virtual reality system? The aspect of ‘presence’ that I wanted to concentrate on was not the illusion of being in a place, but rather that what was happening was ‘true’. I am going to label this phenomenon ‘psi’ (if I give it any actual word, then just like the word ‘presence’ there will be endless debates about the ‘true meaning’ of the word, so here I just use a label). By ‘psi’ I mean the illusion that the events and situation that appears to be happening are what they seem to be, even though the person involved (the ‘participant’) knows for sure that they are not really happening. In other words the participant knows that what is happening is fake, but nevertheless gets drawn in, and finds themselves behaving as if it were true. A very powerful illustration of this can be found in is role playing, but of course it happens to some extent also in theatre and movies. It especially happens in children’s games.

So I wanted to set up a situation where no one could really believe that what they were seeing was ‘true’, but would nevertheless find themselves responding as if it were.

I combined two different scenarios – the Stanley Milgram Obedience experiments (of which I have written before) and the rubber hand illusion.

At the beginning of my talk I asked for two volunteers. One man and one woman were selected from the audience and they came out to the front. Let’s call the man the ‘learner’ (L). I gave him some sheets of paper on which were printed some quiz questions and answers, and I asked him to sit down and learn as many of these as possible. He said “There are quite a lot!”, I told him to remember as many as he could. Meanwhile I was explaining a bit more to the audience. I said that I had only done this experiment “once before”, while he was learning the answers to the quiz questions.

After a few minutes I asked L to sit by a table that had been prepared at the front, on which was a small grey screen and a rubber arm. I positioned his real right arm behind the screen so that it was out of the view of L, and placed the rubber arm where his real right arm would be. I covered with my jacket the space between his shoulder and the rubber arm, so that it looked like his arm was sticking out from underneath the jacket. Then as is the case with the rubber hand illusion, I synchronously touched and stroked the visible rubber hand and the non-visible real hand. In other words L saw the rubber hand being touched, and simultaneously felt his own hand being touched in the same location, but only saw the rubber hand. After a few minutes of this stimulation I asked him if he felt any thing, and he said “That’s my hand!” indicating the rubber hand. In other words it seemed to him as though the rubber hand were his real hand. Then I stopped tapping, and asked him what he felt – he reported that he still felt that the rubber hand was his hand. Meanwhile the audience had crowded round so that they could see more closely what was happening.
The second volunteer I will call ‘the Teacher’ (T). Her instructions were to read out the quiz questions one by one. Whenever L got a correct answer she was to just move on to another question. Whenever he got the wrong answer she was to say ‘Incorrect’, stick a pin in the rubber hand, and then give the correct answer. Each time the answer was wrong she was to leave the pin sticking in the rubber hand for longer. I emphasised that she was only to stick the pin in the rubber hand and not in the L’s real hand! L also emphasised this point.

Then the procedure started. I was standing by L, occasionally reinforcing the rubber hand illusion with some more synchronous stroking and tapping.

After the first wrong answer, T stuck the pin in the rubber hand, and L just shrugged as if to say ‘nothing happened’.

After the second wrong answer T stuck the pin in the hand, waited a short time, and at first there was no response from L, but then suddenly he said “Ouch!” this time he felt something. I reinforced the rubber hand illusion a bit more at that time.

After the third wrong answer (of course he was giving some right answers too) she stuck the pin in longer, again there was a pause, and then he screamed quite loudly. The fourth time he really yelled very loudly, exhibiting many signs of pain.
T said to me that maybe she shouldn’t continue, because L was obviously in distress. I said “Although the procedure involves some discomfort, there is no permanent damage, and that for the sake of the experiment we should continue.” L himself was very reluctant to do one more, and said that I would owe him a pint if he continued. So we agreed to continue. Meanwhile the jacket had fallen off and the rubber hand was simply laying there on the table with no connection whatsoever to L’s body!

The next time after an error L let out one almighty yell. T had tears in her eyes saying that she didn’t want to continue. I was saying that although there is discomfort there is no permanent damage etc.. By now the audience was shouting that we should stop.

At this point I asked everyone to thank the ‘volunteers’ and started clapping and there was a great applause. I explained that they were both actors who had been playing a role that we had planned together earlier. In fact L was Rod McCall, a research scientist at Fraunhofer FIT, Germany, a member of the PEACH committee who I’d recruited the day before, and who had previous acting experience. T was played by Anna Bellido, from the Event-Lab in Barcelona. Both did a very excellent performance.

The reaction of the audience was interesting. No one could have rationally believed that Rod was experiencing pain from sticking a pin in the rubber hand. But I am sure that many had doubts! Afterwards many people told me that they had found the experience unnerving. One person explained that he felt bad about what had happened, because he really wanted to come down to the front and stop what was happening, but didn’t have the courage to do so.

This illusion is an excellent illustration of ‘psi’. Everything conspired together, the serious way that I had announced it and treated it, the fact that I said “I have only done this once before”, the quality of the acting, the fact that people may have not been sure whether the rubber hand illusion extended to pain. Even I, who knew that the whole thing was fake, nevertheless started experiencing anxiety with increased heart beat, at Rod’s screams! This was one reason why I cut the event to a time shorter than I had originally planned.

My contention is that ‘psi’ occurs in mediated experiences – that of course even though everyone knows that the events that are apparently happening are fake, they still respond to some extent as if they were true. Virtual reality is especially powerful, because not only can psi occur (these events are what they appear to be) but there is also the illusion that you are in the same place where they are happening. This is not only happening, but it is happening to you, here directly in your vicinity. So the response is likely to be all the more powerful than if you are simply watching remote events, e.g., on tv.

I see that ‘presence’ has these two components at least – the transformation of the sense of the space that you are in, and secondly, psi, that what is happening in that space is true. But both are illusions and the participant knows that they are illusions. This knowledge dampens down responses compared to reality, but still there can be very strong responses.

RAVE in Barcelona

2007-11-24T11:50:00.000+01:00

I attended PRESENCE 2007: The 10th Annual International Workshop on Presence in Barcelona 25-27th October. It was organised locally by Starlab, who run the PEACH network. The local organisation and especially the design aspects were excellent (as was the food).

Disappointed
About the conference itself – I was disappointed. This series was started in 1998. I found it strange that after 10 of these conferences the number who actually attended in Barcelona (as opposed to those who may have registered) was less than those who attended in 1998. It cannot be location – the 1998 conference was held near Martlesham (Ipswich, UK) - certainly … no less attractive a location than Barcelona. Also, in 1998 there was a spirit, a sense of adventure, excitement, also some level of shared understanding of what we were talking about – ‘presence’ in virtual reality. This year’s Barcelona conference – it was a bit dull. The talks were across such a spectrum of interests and applications that anyone not knowing that this was supposed to be a ‘presence conference’ would not have had a clue – was this a general conference on human-computer interaction, media studies, what?

Endless Debate
I’m not going to speak about the quality of the papers, who am I to say? - especially, strangely and surprisingly, I seemed to be at a conference where this was not … my field. I’m talking about the lack of focus, and I would add the lack in many talks of the kind of precision that is necessary to get a field such as this off the ground.

Anyway, it has come to that point: I don’t think that the term ‘presence’ can be used any more. The days of endless debate about the ‘true meaning’ are over, for me. I’m no longer going to rave about presence. Instead I’m going to rave.

Real Actions / Virtual Environments – RAVE
Why do people smile at an avatar that is smiling at them, when they know full well that no one is there, and no one can see their smile? Why do they become anxious when standing in front of a deep virtual hole in the ground, when they know for sure that there is no hole there? Since the advent of virtual reality in the 1980s it has been well known that not only do people have a feeling of being transported to the place depicted by a virtual environment, but they also tend to act as if they were really there.

This RAVE phenomenon, is at the core of what constitutes my domain of interest. I'm interested in how, within a virtual reality, people act, how they respond, and why. The focus in some particular research may be, for example, on brain imaging, and in another it may be on some aspect of motor behaviour, or the distribution of attention, or emotional responses, etc., or any combination of these. I am not interested in studies that rely on questionnaires or factor analyses of such questionnaire results, and which might normally fall in the area of communication studies, or broadly within the domain of human-computer interaction. The focus is clear: people tend to act realistically in response to virtually generated sensory data. I want to understand why it happens scientifically, and what we can do also as engineers to make it even better.

Also I do not take a narrow view of ‘virtual environments’. Of course there are the traditional means of delivering virtual reality – head mounted displays, Cave systems, large wall displays, and so on. I include also mixed reality, augmented reality. Here the focus shifts slightly, because it is not at all a question of ‘being there’, since you are there in physical reality, only with some augmented sensory data that is generated virtually. So the interesting question in these circumstances is: to what extent do you respond realistically to what is depicted by the augmented sensory data? In this case there are circumstances when the problem is trivial: You see a sign in an augmented reality saying “turn left” and you turn left – nothing much to rave about.

There is also the so-called ‘book problem’ – can you rave about a book? I will deal with this another time, since it raises some issues that are beyond the ideas I want to present today.

This RAVE phenomenon has profound ramifications across many dimension

Neuroscience and Psychology – what is it about the way the brain processes sensory signals that makes it possible for relatively poor simulations of reality to spark such a high degree of realistic activity? How can we use this understanding to design better environments?
Neuroscience of the body - Even the very notion of the human body and our relationship to our own bodies can be transformed. This has very deep implications for the scientific study of body processing and consciousness.
Computer Science and Engineering – given understanding about the psychological and neural basis of this phenomena, how can we build better systems, that maximise the probability for the maximum number of people … to rave?
Applications - to the extent that people show such realistic responses, various fields of endeavour can be approached in radically novel ways: psychotherapy, ergonomics, mission training, industrial prototyping and education to name but a few. When we add the capability for such virtual environments to be shared by many people, we also add a vast range of additional applications, such as remote negotiations and meetings, virtual travel, virtual conferences, and so on.
Philosophy - what are the implications for our notion of reality? Is what we have thought of as reality simply one amongst many parallel realities that we now inhabit?
Entertainment - there are profound new possibilities for entertainment - for example, a person could lead multiple parallel lives - working in the office all day answering emails in “this life”, a private detective in the other “parallel life” within a shared virtual reality. An executive by day, a belly dancer by night.

Raving
In our work in the PRESENCCIA project we have been exposing people to a scenario in which a fire breaks out in virtual reality. They are in a virtual bar, and there are some virtual people there, and eventually there is fire. This takes place in a Cave type system, so that they have the capability to move their whole bodies through the virtual (and corresponding in this case to the real) space. So can we make people run out of the Cave to escape the virtual fire? If not, then no matter what scores they may give in answers to ‘presence questionnaires’ --- they have not demonstrated presence. If they had a ‘feeling of being there’, there in the fire, wouldn’t they run away from it? Well you say, ‘at the cognitive level’ they ‘know’ that there is no fire, so they won’t run. OK, so they are not ‘present’. Maybe they are just a … bit present. Being a bit present is fine, but unless we can demonstrate real actions in a virtual environment, then we have not really demonstrated presence.

It is a worthy scientific goal to try to understand the conditions under which people will run. Surely this is far more fruitful than understanding the conditions under which they will put some tick marks in one place on a questionnaire rather than another, or to have endless debates about the true meaning. Presence, or now I would say rave - means … they run out. This is simple and clear. Sorry for raving on about it.

Obedience in Plaça Espanya

2006-12-28T14:23:00.000+01:00

I was at home in Barcelona - 20th December 2006 turned out to be an interesting day. At about 2pm PST we had a paper published in the new PLoS ONE journal, which recreated aspects of the experiments carried out in the 1960s by Stanley Milgram, on obedience to authority. The day before University College London had issued a press release, with an embargo of 2pm PST, as required by the journal. I was contacted on the afternoon of 19th by the BBC Radio 4 Today Programme, to talk about this paper at 8.30 am (UK time) on 20th.

On 20th I was told by the Today producer that Prof. Alex Haslam of University of Exeter and myself would be interviewed by John Humphreys. The interview seemed to go well, but the BBC insisted on using Skype, and the sound quality on that occasion was not too good, John Humphrey’s even saying “You sound like you’re talking from a virtual world!”.

I received many emails that morning from the press, but I couldn’t attend to them immediately, since I was taking a flight.

We live near Plaça Espanya, and there is a short walk from home to the stop for the airport bus. I was walking towards the bus stop and suddenly an oldish man, a bit pathetic looking, came to me waiving a map. He asked me how to get to Sagrada Familia. I started to explain a bit in Spanish, a bit in English, but he looked confused. Suddenly two other men appeared out of nowhere. They were not well dressed, unshaven, seedy looking. They said that they were police, and that they were following this man who had been talking with me, because he was selling heroin. They asked to see my passport. Many thoughts rushed through my head: I was relaxed since I had done nothing, and could prove everything about who I am and where I was going. They were dressed in that seedy way because they were undercover. I will show them my passport, and quickly be on the way to the bus stop.

I showed them the passport, and also the other guy showed them some scrappy paper for his documentation. They returned my passport, and they asked to see my wallet. They had a quick look, and seemed disappointed. They gave it back to me and asked to see my money, since they said they believed I was buying heroin. I had only 5 euros in my pocket and they saw that and looked disappointed again. They kept asking me whether I had any more money, and suddenly I noticed that the original man who had approached me was no longer there. Where is he? I thought he was the one selling drugs, but the police didn’t seem interested in him. Wait a minute! “Show me your ID!” I shouted at them. This confused them. One of them started to reach into his pocket, but didn’t complete the move. The other one eventually pulled out a stupid looking card inside a transparent wallet. “Hmm, it doesn’t look too realistic” I said. “If you don’t believe we’re police we can take you down to the station.” OK, I thought, and why are they speaking with East European accents?

This is called a ‘break in presence’ – a moment when the illusion provided by the virtual reality breaks down, and you find yourself where you really are – in the case of virtual reality you would be standing in a computer science lab, wearing a head-mounted display or some other display and tracking devices. OK, in this case, it wasn’t a virtual reality, but a small drama, like being suddenly in a play. The dramatic reality attempted by these three men suddenly vanished, and there I was in Plaça Espanya, with two seedy-looking men conning me, for what exact purpose I wasn’t sure.

I walked away rapidly, shouting “This is a setup!”. The look on their faces said “You got us!”. I wondered about phoning the police, or trying to find a real policeman. At the airport bus stop there happened to be two motorbike police. I told one of them what had happened, and he was most concerned to know if I still had my passport (which I did). He said he would go to find them, but they had been on the other side of the plaza, and by now must have escaped.

So on the very day our paper that described a reproduction of aspects of the obedience experiment in virtual reality was published, there I was having my own experience of obedience in a dramatic reality. So what happened? Why did I obey, and show these people my passport and wallet? What lessons can we learn about ‘presence’?

Well there was a smooth transition from ‘reality’ to this drama. I was doing something entirely natural – walking along in the street. A man asks me directions, he is speaking to me in a broken English. The two other men appear with a plausible story. The fact that the first one had spoken to me in English somehow I didn’t notice that these two supposed undercover police were also talking in English. Because of the whole situation, I didn’t notice that their accent was not Spanish. There are small transitions, small changes that go from normal physical reality to this dramatic reality so that the two coexist. Then there is a contradiction – if the first guy was a heroin seller, and these are police, why have they let him disappear from the scene? Why are they more interested in me than him? Then the whole illusion falls away at once – I notice their clothes and other aspects of their appearance that just doesn’t fit with the story, and also I notice now their accents. The illusion is completely stripped away, and you see the other (every day) reality shining through. So the transition from real to ‘virtual’ is gradual (until the contradiction that breaks presence in this dramatic reality).

Why obey? During the moments that the story was believable, the overriding thought is that being innocent if I cooperated I would quickly be on my way. The very fact that this idea was inconsistent with the unfolding events (they kept asking me questions even though I was obviously cooperating and obviously innocent) was a contributory factor in breaking the illusion and making me suspicious. Of course, it was a different situation from the Milgram experiment – there subjects were asked to carry out an act that would harm another person, against the wishes of that other person, and against their own inclinations. Here it was a question of obeying what appeared to be a legitimate authority (for a few seconds) and also an underlying implied threat of force should I choose not to obey. But still I showed my wallet to those people!

There are after effects to these kinds of incidents. One is a (very) mild ‘post traumatic stress’, which is mentally going over the incident again and again, repeatedly asking: Why did I show my wallet to those people? and also feeling some anger. Also the incident has another effect, a blurring between real police and these ‘actor’ police – a mild fear that the police are now somehow a threat to me. This dramatic reality, only lasting a few seconds, is very powerful: as we said in the paper about the Milgram experiment – even though nothing real happened, there is part of me still responding as if it had been real.

Ground Truth in Zurich

2006-11-29T18:46:00.000+01:00

And believe it or not, soon after returning from Cyprus I had to go to Zurich. This was to attend a meeting of the European ‘Future Emerging Technologies’ project called IMMERSENCE. The timing was fortunate since my wife also had to give a seminar in Zurich on the same day (yes, she is a scientist). The project mainly consists of experts in ‘haptics’. Haptics is concerned with touch and kinesthetics including force-feedback. Normally of course if you are in a virtual reality and you touch a virtual object you don’t feel it (because nothing is there). Remember that the stereo and head-tracking system may give you the illusion that an object is there right in front of you, but reach out to it, and you will feel nothing. This can (at least temporarily) break presence. Haptics typically employs devices that give back some feeling – normally the force feedback you get on touching something (i.e., when your hand collides with a solid object it does not pass through it) but less often the associated tactile sensation.

My role in the project is to investigate the extent to which presence is induced in a number of paradigmatic application scenarios. In this I work with the researcher Andreas Brogni in the group at Universitat Politècnica de Catalunya in Barcelona.

The idea we follow in this project is to compare responses in physical reality to those in virtual reality. For example, if in physical reality you pick up a cup, you will have applied forces, used your hand and arm in a particular way, felt various tactile sensations, resisted against the weight, and so on, all the way through to having various thoughts and feelings associated with lifting that cup. Now when you carry out the same task in a virtual reality, to what extent do you carry out the same actions, and have the same overall responses to your own actions and to the totality of the event ‘picking up the cup’. Presence in the virtual environment is the extent to which your responses are the same as those in the real environment.

Let’s consider this approach to presence a bit more. It is very practical. Ontological considerations can be avoided. But … many people have said that it does not take into account the beauty of virtual reality which is able to create experiences that are not real. For example, in virtual reality you can ‘fly’, there is no naturally occurring virtual gravity that will hold you at virtual ground level. It is all a matter of what the program allows you to do. For example, it is trivial to program the system such that if you look upwards and press a button on the wand (6 degree of freedom ‘mouse’) that you will typically be holding, then you can fly up in the virtual world. And why not? So can we not talk about presence in unreal situations, but only in relation to simulations of physical reality?

There are two answers to this. The first is that if in simulations that are bound to physical reality we learn how to increase the probability that the participant will respond to virtual events and objects as if they were real, then of course we can apply the same knowledge to non-physically grounded simulations, and increase the chance that people will respond to those types of situations and events as if they were real.

But the real answer is that this approach to presence does not demand at all that the virtual environment depict something that could be real – it is only that the responses to the virtual environment should be real (high presence). In other words if you learn to fly within a virtual reality and you respond to that experience as if what you were doing were real, then that is …. presence. Of course, in such situations the hard part is that there is no ground truth against which to compare – we do not usually fly within physical reality (the plane flies, we are just along for the ride). But what we do in virtual reality is expand the range of experience, to know something about what it would be like to fly - with realistic responses helped along by the exploitation of knowledge we have obtained by studying situations in which there is a ground truth.

Action in Cyprus

2006-11-14T09:35:00.000+01:00

Then almost immediately after returning from California I had to go to Cyprus (Limassol) for a conference called ‘Virtual Reality Sofware and Technology’ (VRST 2006). What happened was that it turned out that I was one of the Chairs of this conference – apparently I had agreed to this but forgot. Then I thought – well if I’m Chair I will have to go, so we submitted a short paper.

The paper was about the impact of different types of rendering quality on ‘presence’ in virtual reality. What do I mean by rendering quality? – well in computer graphics you might display the scene just with lines (called ‘wire frame’) or with flat shaded polygons, or with textured polygons … or ultimately with objects that appear to be shaded correctly according to the light distribution in the scene. The latter is very hard to do – and until recently was impossible to do in real-time. But we had a way to do a limited version of this ‘global illumination’ – based on ray tracing - and the question was: what difference would it make?

You might think that the answer is obvious – the more realistic that a scene looks the more likely it is that people will feel themselves to ‘be there’. But actually the evidence to date doesn’t support that idea – there have been experiments that suggest that it makes no difference (google Zimmons rendering quality). The difference with our research was that we not only had a level of global illumination, but one where the shadows and reflections actually move. We had people looking down over a virtual pit – and they would see the shadows and reflections of their virtual body move in correspondence with their real body (well, to some extent, since we only can track their head and one hand). Although this experiment was completed a few months ago, to date I’ve only had time to analyse the questionnaire data – there are many questionnaires that attempt to measure presence and we used one of these. I call the type of presence that you can get from the use of a questionnaire ‘reported presence’. So our study showed that ‘reported presence’ seemed to be greater on average for those people in our experiment who experienced the pit room with the ray tracing rendering method (which has shadows and reflections) compared to those who experienced a method of rendering that did not include second order lighting effects (i.e., shadows and reflections). It seemed to make a difference. But is it only because there is dynamic feedback, or is it because it simply looks more realistic, or both of these? We have to wait for the next experiment.

But if there’s ‘reported presence’ is there also … unreported presence? Why ‘reported presence’ and not just ‘presence’? Well actually there is ‘unreported presence’! For example, more than 10 years ago when I was at Queen Mary College and we had one of the first types of Head-Mounted Display we had some London firefighters in for a demonstration. The environment we showed them was the pit room. There were two of them, and when they noticed the virtual pit, each of them visibly shook, and stepped back – they had a very strong behavioural response to what they were seeing. But … when we asked them afterwards about their experience (informal ‘reported presence’) in particular whether they had any sense of ‘being there’ the replies were … ‘Oh no mate, felt nothing, no nothing happened’. Their words contradicted what we saw with our own eyes.
Presence as far as I’m concerned is quite uninteresting if it is limited simply to what people will tell you about their experience after the event – no matter how well ‘validated’ the questionnaire; and you can’t ask them during the event because that in itself could destroy the experience. So presence is to do with how people respond to events and correspondingly how they are able to act within a virtual reality (or even in a mixed reality) – it is their response and activity in relation to virtual sensory data (how exactly it is produced is not important – but if you want to think of it as computer generated on computer controlled displays that is fine).

Obtaining their answers to some questions after they have completed the experience is ok, and can be useful. But what is more important is how they respond at many different levels: their physiological responses (e.g., heart rate and heart rate variability, temperature, electrodermal activity – skin sweat response – even measures of brain activity such as EEG). Then there is their automatic behaviour – like do they duck automatically if a virtual object flies towards their face? Then there is deliberate behaviour – like they decide to pick something up and look at it. Then there are thoughts ‘I wonder what’s going on over there?’ and emotions…. Basically the total response of the person to the situation and unfolding events. (And this is why the ‘pit room’ provides such a useful scenario for testing people’s responses within a virtual reality – since we know the types of response that people would normally have in reality).

So in virtual (or mixed reality) we substitute (or augment) real sensory data with virtual sensory data. The question is whether people respond to this as if it were real.

And … obviously there are different levels of this ‘presence’ – since as we saw in the firefighters example, they may exhibit high ‘presence’ in one field (automatic behavioural in that case) but low ‘presence’ in another (reported presence). High presence means that they are ‘high’ across the board.

Presence - the view from Marina del Rey

2006-11-06T00:17:00.000+01:00

During the last week of October I was at the Institute of Creative Technology at University of Southern California in Los Angeles (Marina del Rey). The purpose was a meeting of invited people only to discuss presence in virtual reality, but where there was a mix of computer scientists, human factors people, engineers, artists and people involved in computer games. The meeting was interesting, and a great idea, but also I found it very frustrating - although there was a great view outside my hotel room.

This concept of ‘presence’ has been around for a long time – it started with ‘telepresence’ in the context of teleoperator systems (people controlling remote robots) and then shifted to ‘presence’ when virtual reality came into vogue in the late 1980s early 1990s. This is not going to be an academic article so I’m not going to provide any references – take a look on ‘google scholar’ and look up ‘presence’ ‘Durlach’ ‘Held’ ‘Sheridan’ ‘Loomis’ for some of the early discussions.

Basically presence has been thought of as ‘the sense of being there’ – some people when they go into a virtual reality experience a shift of their sense of place, so that they feel themselves to be in the place simulated by the VR rather than in the physical place in which they actually are.

This is the approach that I used also for many years, and it was useful for a while. But, how do you find out if someone has a ‘sense of being there’? The only way of course is to ask them, and that leads to the use of questionnaires and interviews. Anyway, over the years there have been a large number of papers devoted to this issue, there is an International Society for Presence Research (ISPR – don’t confuse it with the International Society for Paranormal Research), and several groups across the world who contribute to the ‘presence’ literature.

But…..

After all these years, a large part of the meeting at ICT was devoted to discussing ‘what presence really is?’ searching for the ultimate ‘definition’ of presence.

When you hear this kind of thing in discussion of a research subject, you know that the game is already lost.

How can there be ‘presence research’ when no two people seem to agree what they are talking about?

In spite of all the definitions of ‘presence’ - what presence ‘really is’ is equivalent to what ‘presence researchers’ do to measure it. Someone might define presence to be X, but if you look at what they do it may or may not have some relation with X. I was very frustrated during this meeting and the fruitless discussions about ‘definition’ – not too useful or exciting decades after people first started researching into this subject.

So I decided to start this ‘blog’. In this blog I’m going to try to explain some of my ideas about this phenomenon, but more importantly what I try to do to define it in practice – as I said what presence really is for any researcher is exactly equivalent to how they attempt to ‘measure’ it. So what I do is more important than what I say.