Technology as extensions of our selves – but what’s actually extending?

Jim Makes a Frittata in His New Frittata Pan
Jim Makes a Frittata in His New Frittata Pan

Some initial thoughts on human-computer interaction after several weeks of intense literature review on ‘virtual embodiment’.

I was eating a frittata the other day – if you don’t know what a frittata is, that’s OK, doesn’t matter to get the idea (but it’s that thing up there in the picture).

Anyway, while I was taking the last piece from the oven casserole (which is made out of material that feels like porcelain or something), I used a knife to gently scratch some crispy cheese from the corners of the pan. A sudden, annoyingly simple idea came to me: with the stainless steel dinner knife, I was able to feel the material of the casserole, and judge from that if I would scratch and ruin it or not.

While thinking of this, I also tried the pan surface with my fingers. The feeling I got from that pan was basically the same as with the dinner knife.

Now, this might feel rather arbitrary and useless pondering of everyday life, but from such little realisations often come the most interesting things. We are so used to such experiences without thinking how they occur to us that we don’t always think how they might take place in other contexts, such as in using virtual environments.

If I am able to get the feeling of a material or a quality through another object, such as the feeling of a frittata pan through a dinner knife (similarly perhaps how a tennis player experiences a ball through a tennis racquet or a crane operator feels a crate through the crane), what various things do I get a feeling of through interactive media input-output technology? It seems that authors and others like to jump on board with McLuhan’s (1964) term ‘an extension’, but do we extend our thinking beyond that term (which itself is a tool of language to communicate about these matters)? What actually does extend?

So, what different matters or qualities might get extended through control devices such as game controllers? As a gamer, I can reflect that for example in Halo games, different weapons and vehicles feel different, and I prefer some over others – I loved the double needler when it was available. Nothing could give more satisfaction than blowing someone away with them with a huge explosion.

At the same time, interactive media extends our sense of social presence. Now, this is a different matter altogether. Technology gives us the possibility to see another person in the form of a virtual embodiment, an avatar. How the avatar looks like, what it does and how it communicates transmits us through a screen that there is actually someone there. It extends social interaction, and the social interaction is as real as it is in “the real”. The growing normality of this experience is blurring the fact how unique that is in the evolution of human-technology interaction.

As a sub note, I am developing a rather big issue with terms such as virtual and real. I feel Rob Shields (2002) has done a nice job in trying to get us out from this definition muddle. If we continue to talk about ‘virtual’ as something less real than the concrete world around us (which also psychologically and philosophically is very much debatable) we are basically saying virtual experiences are not real. This would mean that our experiences in various virtual worlds, such as the long grinding sessions in WoW or Borderlands never happened. I am sure no reasonable person would give such a comment. So why do we continue saying experiences such as social interaction in virtual contexts are somehow less than face to face? Based on some of the studies I have examined, for some users some forms of experience can be even be more.

Thinking such things tend to gather sort of blank stares and comments like “well, that is very philosophical, but…”. I disagree with the “it is so philosophical” (whatever it might mean for some people to say this). I think it is rather pragmatic to use your head to try to see what these experiences really truly mean instead of jumping on the technology hype or the luddite movement. It lifts the veil over the novelty of emerging technologies and sort of re-focuses thinking to how we actually interact with objects and other people.

(in)Fidelity, interaction and virtual reality: With some feeling

Blue Eye Soup

Image by Richard Paterson

This is a collection of thoughts, or sort of a “commentary track” to a recent post in Verge (

As it is reported, Valve has been designing VR hardware together with HTC.

The HTC Vive (rhymes with “hive”) headset…builds on years worth of virtual reality research, focusing specifically on creating an experience that’s ambulatory and interactive.

While Oculus is still coming up with a motion control system that’s worth showing off, Valve has made visible progress in interactivity — something that, more than any boost in graphical fidelity, is what we need to spur new, innovative uses of virtual reality.

I will not go into the language of this post because I think it’s quite obvious that the contemporary news media is under harder deadlines nowadays. I also will not comment the fact that similar kind of a hype, but also actual research was with data gloves already way before. But coming back to the comments, especially the bolded part. It makes a valuable point which seems to be missing from a large deal of academic papers I’ve been read lately.

What still sneaks up to amaze me is the gun-ho irresponsible use of the terms fidelity and interaction. It seems these terms are used however one feels like. I may have created a virtual environment, and although I do not allow my users to actually use it (interact with the environment by navigating its environment, manipulating its objects, and perhaps communicating with its characters or NPCs), I can still state they are interacting, that it is an interactive environment or a game.

Well boo-hoo. Such people are wrong. Real, embodied interaction with virtual environments assigns agency to the users. These are not restricted school classes where the high and mighty Instruktor owns the ultimate power over the students (the users) who sit and merely pay attention (i.e. are on the verge of sleep).

And the same goes with fidelity. Sorry if it is stating the obvious, but who invented that ‘fidelity’ was the same as representation or “better graphics”? When did this happen? If we take the good ol’ Merriam-Webster from our virtual shelf, and see what fidelity means (beyond the “staying true to your hubby or companion”) we get something like “accuracy in details”, “exactness”, and “the degree to which something matches or copies something else”. There is a lot more to copy in reality than just the graphics. Can you name one, two, fifty?

Now, if we were just some hovering eyeballs without any body, we might say representation is the only thing we have (although I would still disagree). But we aren’t. We have a thing called body, and we experience as embodied creatures.

You can easily try this out with empty virtual worlds or environments. For example, do a random visit to Second Life or try one of those graphics demo test thingies that put your PCs GPU on its knees and where you can nothing but move around and look at things: they are awesomely booooring. You don’t have much to do, no one or no thing to interact with, and no time-story continuum – OK, you might build all of this in Second Life (with patience), but that’s another story altogether.

But alas! A saviour on a white stallion and shining armour has come to the fore to build better theory. I really enjoy latest writings by Gordon Calleja who has started making some sense to this field. His theoretical contributions aim for better understanding in human-virtual environment relationships and user experience. I like his effort of using a new concept, incorporation, in order to root out some old terms such as immersion and presence.

He says thinking virtual is “there” and real is “here” is already a fundamental flaw when thinking human relationships with virtual environments. He suggests we should focus on the agency given to us to navigate and interact (to inhabit) a virtual environment, and that it inevitably takes place in our consciousness. It is an active human-computer interaction process, where we become involved with the virtual (if we feel like it, that is) and the system hopefully sustains our willing participation. I’ve played so many games with awesome graphics, but if something is not right, the missions, story or characters are boring, or the playability and usability is crap, I will opt out never to return.

Unfortunately, this just might be the major problem with at least VR applications for professional use: design-wise for such incorporation, they are merely hollow prototypes that are not able to sustain such interaction.

Read more:

Calleja, G. (2014). Immersion in Virtual Worlds. The Oxford Handbook of Virtuality.

Facepalms and Lulz: Blind spots in immersive virtual environment research reports in safety training contexts

Image from

Lately I’ve been gathering a literature review in the use of immersive virtual environments (or IVEs) in safety training contexts – this might include construction, mining, military, healthcare etc. A couple of quick observations below, in a non-academic and tightly socially acceptable way – although one could go beyond that.

  1. Terms such as virtual reality, immersive virtual environments, virtual worlds, simulations, serious games, immersion, and presence (a non-exhaustive list) are used however one feels like – nope, no need to justify them. You might have a virtual environment with which you don’t even allow the user to interact, a regular monitor as the output, and you call it a ‘virtual reality’ and a ‘serious game’. A double whammy! Hmm, I really need to bend my head in so many ways to agree with this that I am actually getting an idea: a journal article!
  2. It seems people still have the misconception that virtual environments are about representation. Interaction seems to be something like the evil twin that everyone wants to hide, or pretend it does not exist. Don’t these people play video games, like really?
  3. No one is interested in the actual user experience – oh yeah, that too exists! If you ask “did you like it” and they say “yeah, it was good” I don’t think it should cut as genuine research. At least I don’t have any idea what to do with such comments. But…
  4. …everyone is interested in fidelity, life-likeness or realism (within the little world of object number 2 on this list). How using different input/output devices, level of interaction with the environment, basically the whole shebang of human-computer interaction and user experience is tossed aside in the name of understanding better graphics! I am sure old MUD players would at least now facepalm, if not already before. Immersion has been there for ages, and assigning it all to better graphics is just denying the existence of everything else.

Just a couple of thoughts.