Thursday research bulletin 16.7.2015

So this week’s been pretty much packet mainly with different kinds of research articles. There’s especially the mentioned Curtin Business School Colloquium article in the area of HCI, virtual embodiment and human-technology coupling, which I pretty much re-wrote although I had some of it ready from before. I guess part of research “rigour” (the word we all love) is that you will re-do everything if you feel something is not right – even if you don’t feel like it. Anyway, it looks much better now and perhaps I can extend it to a “real” journal article somewhere along the way.

From the Web

In the VR section, I guess one of the interesting things this week was HTC announcement that finally something actually tangible might come out in October:

I’ve also followed Leap Motion’s virtual hands concept with an interest:

Naturally I would like to explore it with Talos Principle which I’ve recently played:  

I guess in the section of embodiment and “Oh well, it’s only a year or two when Tony Stark’s gadgets start to look old”, this post sort of made the day:

Technology as extensions of our selves – but what’s actually extending?

Jim Makes a Frittata in His New Frittata Pan
Jim Makes a Frittata in His New Frittata Pan

Some initial thoughts on human-computer interaction after several weeks of intense literature review on ‘virtual embodiment’.

I was eating a frittata the other day – if you don’t know what a frittata is, that’s OK, doesn’t matter to get the idea (but it’s that thing up there in the picture).

Anyway, while I was taking the last piece from the oven casserole (which is made out of material that feels like porcelain or something), I used a knife to gently scratch some crispy cheese from the corners of the pan. A sudden, annoyingly simple idea came to me: with the stainless steel dinner knife, I was able to feel the material of the casserole, and judge from that if I would scratch and ruin it or not.

While thinking of this, I also tried the pan surface with my fingers. The feeling I got from that pan was basically the same as with the dinner knife.

Now, this might feel rather arbitrary and useless pondering of everyday life, but from such little realisations often come the most interesting things. We are so used to such experiences without thinking how they occur to us that we don’t always think how they might take place in other contexts, such as in using virtual environments.

If I am able to get the feeling of a material or a quality through another object, such as the feeling of a frittata pan through a dinner knife (similarly perhaps how a tennis player experiences a ball through a tennis racquet or a crane operator feels a crate through the crane), what various things do I get a feeling of through interactive media input-output technology? It seems that authors and others like to jump on board with McLuhan’s (1964) term ‘an extension’, but do we extend our thinking beyond that term (which itself is a tool of language to communicate about these matters)? What actually does extend?

So, what different matters or qualities might get extended through control devices such as game controllers? As a gamer, I can reflect that for example in Halo games, different weapons and vehicles feel different, and I prefer some over others – I loved the double needler when it was available. Nothing could give more satisfaction than blowing someone away with them with a huge explosion.

At the same time, interactive media extends our sense of social presence. Now, this is a different matter altogether. Technology gives us the possibility to see another person in the form of a virtual embodiment, an avatar. How the avatar looks like, what it does and how it communicates transmits us through a screen that there is actually someone there. It extends social interaction, and the social interaction is as real as it is in “the real”. The growing normality of this experience is blurring the fact how unique that is in the evolution of human-technology interaction.

As a sub note, I am developing a rather big issue with terms such as virtual and real. I feel Rob Shields (2002) has done a nice job in trying to get us out from this definition muddle. If we continue to talk about ‘virtual’ as something less real than the concrete world around us (which also psychologically and philosophically is very much debatable) we are basically saying virtual experiences are not real. This would mean that our experiences in various virtual worlds, such as the long grinding sessions in WoW or Borderlands never happened. I am sure no reasonable person would give such a comment. So why do we continue saying experiences such as social interaction in virtual contexts are somehow less than face to face? Based on some of the studies I have examined, for some users some forms of experience can be even be more.

Thinking such things tend to gather sort of blank stares and comments like “well, that is very philosophical, but…”. I disagree with the “it is so philosophical” (whatever it might mean for some people to say this). I think it is rather pragmatic to use your head to try to see what these experiences really truly mean instead of jumping on the technology hype or the luddite movement. It lifts the veil over the novelty of emerging technologies and sort of re-focuses thinking to how we actually interact with objects and other people.

Research imitates (science) fiction, or which way around is it?

Lately I’ve been trying to get to the core of things around virtual reality and the metaphors that are commonly used when describing the experience of being immersed in computer-generated virtual environments such video games. The deeper I get in comparing the virtual and the actual, the more fuzzy it gets.

Interestingly, it seems to be the case that many current definitions about cyberspace or somehow explaining the virtual experience rely on a single book, namely Murray’s (1998) Hamlet on the Holodeck. In her book, she describes virtual reality through the Star Trek holodeck concept. A holodeck is a room that transports its users to different times such as old London, and creates tangible objects and natural characters who notice your presence and interact with you. All of this seems like real.

I have to assume Murray intended the holodeck to be a metaphor, and not analogy as people seem to have started to treat it. If you believe we humans have a an actual body and that we are even little bit a bodily construction instead of just floating brains, in no way can we “go” to a virtual environment. What tangible concrete part of us would actually go there? And if we went there, where in the bytes was this “there” anyway?

For example Jesper Juul (2005), Salen and Zimmerman (2004) have challenged this very notion of transporting to virtual, which as a starting point for research in my eyes skews studies perhaps slightly towards some odd directions. Concepts such as presence emerge, that try to explain our experience in the digital realm with the terms of our physical experience. Some make comparative studies and find odd results that state we experience more presence in the virtual than the real. Is there a problem in the instrument or is there actually something in this?

I guess that’s the skewness of the thing. As I’ve confronted in several cases, virtual should not be considered as the opposite of the real (e.g. Shields, 2003; Lévy, 1998). As a light note, if it was, the logical outcome would be that there’s no use to get university to fund any research in that domain: “I am researching the virtual. The virtual is the opposite of the real. This means virtual is non-real.” How can you study something that does not exist?

I’m sure many would consider this kind of pondering (which used to be called ‘thinking’, ‘logic’ or even something nowadays rather underrated called ‘philosophy’) rather useless. But as mentioned, I think it is the core of things. I somehow consider this whole ‘presence this and presence that’ discussion rather odd. For example, whenever I go to see a boring lecture, I might be physical present but actually very absent. On the other hand, playing Halo Reach, I might feel really ‘in the game’. So what is the use of this kind of physical presence thinking then? I agree with several authors like Gordon Calleja (2011) that we need to define clearly what we are talking about, and also open up the ontological basis of our definitions, and not assume that more naturalistic accounts are the only ones.

Coming back to Murray’s holodeck as a transportation to a virtual world. Recently I read a very interesting article by Dourish and Bell (2014) called “Resistance is futile”: Reading science fiction alongside ubiquitous computing”. Basically their article contrasts popular accounts in science fiction to actual technological developments and research literature. It is interesting to notice that there is this kind of a back and forth movement between fiction and real studies, sometimes even to the point of seeing them merging.

As it is long known e.g. in media studies, stories are a (virtual?) place for humans to ponder many levels of developments from culture to technology and so on. Such themes ooze to research literature. What makes this interesting, is that they are often taken for granted metaphors that come together in simple terms such as “immersion” or “presence” – and often very revealing about how their users preconceive what constitutes the world for us.

But hey, of course everyone knows what terms such as “presence”, “immersion” or “embodiment” mean! Well, go and ask 5 people near you. I’m sure you’ll get 5 different answers – and of course how you asked varies that even further. Even in studies, we seem to stick with our implicit ontological views which are affected by popular fiction metaphors from sources like Neuromancer, The Matrix or more contemporary Sword Art Online. These accounts fuzzily combine concrete real (e.g. existing social structures) with possible real (humans can put a VR helmet on and ‘actually’ live in a VR world). What amazes me is how are we capable of maintaining a solid research body in things virtual, when our very foundation is build on such quicksand?

Check these out:

Dourish & Bell: Resistance is futile: Reading science fiction alongside ubiquitous computing. Retrieved from http://socialcomputing.uci.edu/system/files/publications/puc-scifi-draft.pdf

Juul, Jesper: Half-Real. Book website at http://www.half-real.net

Lévy, Pierre: Becoming virtual.

Murray, Janet H.: Hamlet on the Holodeck.

Salen & Zimmerman: Rules of Play (especially “The Immersive Fallacy”).

Shields, Rob: The Virtual.