Testing Room-scale VR

I tested the HTC Vive headset recently which opens up a whole new realm of possibilities. This obviously means more ways in which issues can manifest.

The HTC Vive is geared towards a Room-scale VR experience. The user will be generally stood up during use. This means more things need to be taken into account by the teams developing and testing these applications, which are mostly games at this point.

Now although Oculus Rift have stated that their headset is capable of Room-scale VR it is geared towards a sat-down experience. Understanding these distinct experiences is necessary before we can start to think about how we are to test them.

Sat down experiences take out the issue of a user bumping into their environment. It would still be possible in some particularly confined setups, but these are edge cases IMO (still not to be fully dismissed). Sat down experiences also take out the worry of becoming entangled in the headset cable and take out the possibility of a user tripping over.

Whilst all the above is true, sat down experiences also lose an element of immersion. Putting the HTC Vive headset on and experiencing the environment for the first time is an incredible feeling. Something as simple as kneeling down to pick an object up feels special. The controllers for the Vive also add to the experience, but they also provide another area for things to go wrong.

Applications being developed and tested for Room-scale VR have to take into account the variety of spaces people will have. Some developers have stated that you don’t need a huge amount of room to experience Room-scale VR. It is said you need enough room to stand up and stretch out in all directions. This video from the makers of Hover Junkers explains:

Whilst the makers of Hover Junkers have been very attentive to room size issues and the range of rooms users will have; we cannot assume everyone else will. We have to be aware that room size issues will come into play. Using boxes/crates to quickly change the test space you have is going to be necessary. It’s all well and good having the great test space you’ve setup in the office but how will that translate to the student dorm room?

Thinking about testing Room-scale VR leads me to think that we need a new heuristic to aid this testing.

An interesting discussion and more on simulator sickness

I’ve just watched this very interesting discussion on VR gaming here. I highly recommend watching it. There are a number of concepts discussed, one of which I want to raise here.

So according to Oculus Rift; Simulator Sickness is cumulative. Let that sink in for a minute…

This has a number of connotations that both developers and testers will need to be aware of. It means very slight triggers will build up over the course of the game and could overwhelm the user. The general advice to users is that if you feel any type of sickness, then you should take off the headset and have a break. Now I know that bit of advice and would generally always follow it, but let’s paint a scenario.

User is playing a game in VR and there are some minor ways in which Simulator Sickness is being triggered. Maybe it’s to do with the way doors have to be opened, but a lot of users generally won’t react to a minor feeling of Simulator Sickness, especially if they’re engrossed in the action. If this is allowed to build up, then it could overwhelm the user and make them feel much sicker than if they’d encountered one thing that triggered Simulator Sickness very quickly.

I encountered my worst experience of Simulator Sickness when I allowed it to build up. I ignored it because I was having too much fun, silly I know but these are the things users do! If you’re immersed in an incredible experience then many people will delay pausing that experience to take a break. This is why it’s falls on the developers and testers to make experiences which minimise the potential.

Simulator Sickness is one aspect of the VR experience that we have to be aware of and account for, amongst many others that I will continue to talk about.

Questions for the new frontier

There are many exciting things about working on VR projects at the moment. The infancy of the technology coupled with the potential of applications is staggering to me. I love the conversations that get bounced around the office. Any conversation can yield questions that no-one on the planet knows definitive answers to (yet).

That is something which completely excites me.

So after checking out some different games in Oculus Rift recently; I gave myself a case of simulator sickness. Now if you’ve never experienced this; it’s similar to travel sickness but can come on suddenly without warning. It can also pass just as quickly for those of you who might be worried.

I was taking a walk around the office and a dev mentioned simulator sickness would be less of an issue; once I had built up a tolerance. Now I’m not sure building a tolerance as a tester is beneficial. It is easier to gauge the reaction of someone with a tolerance to VR systems, but not as easy to gauge the reaction of a new user if you already have a tolerance.

Then the magic conversation. We start thinking about how this tolerance works. Is this a tolerance which is fluid? Given enough time your tolerance could reduce to a lower level maybe? Or is this a situation where once you’ve walked through a gate; there is no turning back and returning to the way you previously were.

There are so many disciplines colliding here, and so many things to think about. I am hyped to see this all unfold as we find out more!

The language we use

A recent debate on Twitter bought an interesting idea to light. The idea that the language used by testers can be separated from ‘testing’. The argument goes,

I don’t want to get hung up on language, I just want to concentrate on testing.

Taken at face value; it’s a reasonable view. Let’s cut the talking, it’s all about the testing.

I don’t think this is feasible. The language we use as testers; is central to what we do and shapes the testing itself.

As is usual in my posts; lets take an example from classic Sociology to illustrate this point.

Becker discusses Labelling theory in his book, Outsiders: Studies in the Sociology of Deviance. Becker says that at one time or another most of us break the law, but only some of us are ever labelled as criminals. Becker says that once labelled as criminals; this not only changes the way that society treats these people, but also how these people see and treat themselves.

So let’s apply some of this to a common issue within our field. Use of testing tools. Now already you can see I’ve started the conversation by calling them testing tools. The language used informs you as to my view of the matter.

These tools are commonly referred to as automated testing. Now many of us have interacted with manager-type people who may say something like,

Can’t we just automate all our testing

In the head of the manager there is a picture that looks like this.


Labelling the use of testing tools as ‘automated testing’ has knock-on effects.

Those within testing understand that use of automated testing tools isn’t a magic bullet. The language used gives the impression that an automated procedure is an easy procedure. It’s an understandable reaction. There are many fields in which automating procedures have made things very easy. However, the same thing isn’t true within our industry.

Using automation tools doesn’t make things easier, it’s just a different kind of difficult.

Now let’s think about what would happen if the term ‘automated testing’ was never used. The manager wouldn’t have in mind the magic automation robot finding every bug. The picture in mind would be similar to any craftsperson using their tools.

The language we use has repercussions in many ways and for a species which uses language as our primary means of communication; it isn’t something we can easily separate from anything else we do. It is inherent.

The way we talk about testing is part of our testing.

Immersion and presence – Why are they important?

Testing is about gaining knowledge. To understand how to test VR effectively; we need to understand VR. In my last post I referenced a paper by Daniel R. Mestre; in this post I will go into what I’ve learnt from this.

So how do immersion and presence work together in the VR experience?

Presence is defined as the sensation of being in the virtual environment

We can think of presence as being a psychological quality. It is our perception of existing inside the virtual environment, it is subjective.

Immersion is capable of producing a sensation of presence

(Ijsselsteijn & Riva,2003)

Let’s think about this connection. Presence is the subjective feeling of being within a virtual environment, and immersion provides a vehicle for this feeling.

“The term immersion thus stands for what the technology delivers from an objective point of view”

(Mestre, 2005)

The connection should be clearer now. Presence is a subjective term; which covers how a user feels about the virtual environment; from a psychological point of view. Immersion covers what the technology can objectively deliver; to give the user a strong feeling of presence within a virtual environment.

Now who is best placed to “measure” immersion levels?

Well obviously I’m going to say testers. We’ve been doing something like this for years, but calling it user experience. Now I’m not saying testing VR is just UX testing, but it is about taking some of those principles and applying it to VR.

We cannot fully control how present a user is within virtual environments, but we can control how immersive a virtual environment can be. If we create an experience which allows complete immersion, then a user is more likely to feel present there.


References from the paper “Immersion and Presence” by Daniel R Mestre


Presence vs. Immersion

This article was bought to my attention. It talks about the concepts of presence vs. immersion, how they relate and cover different aspects of the VR experience.

I’m going to dive into this over the next few days and see how this knowledge can help improve my testing approach.

I’ll be back next week with an article covering what I learn.