Tag Archives: context-driven testing

The language we use

A recent debate on Twitter bought an interesting idea to light. The idea that the language used by testers can be separated from ‘testing’. The argument goes,

I don’t want to get hung up on language, I just want to concentrate on testing.

Taken at face value; it’s a reasonable view. Let’s cut the talking, it’s all about the testing.

I don’t think this is feasible. The language we use as testers; is central to what we do and shapes the testing itself.

As is usual in my posts; lets take an example from classic Sociology to illustrate this point.

Becker discusses Labelling theory in his book, Outsiders: Studies in the Sociology of Deviance. Becker says that at one time or another most of us break the law, but only some of us are ever labelled as criminals. Becker says that once labelled as criminals; this not only changes the way that society treats these people, but also how these people see and treat themselves.

So let’s apply some of this to a common issue within our field. Use of testing tools. Now already you can see I’ve started the conversation by calling them testing tools. The language used informs you as to my view of the matter.

These tools are commonly referred to as automated testing. Now many of us have interacted with manager-type people who may say something like,

Can’t we just automate all our testing

In the head of the manager there is a picture that looks like this.

robo23

Labelling the use of testing tools as ‘automated testing’ has knock-on effects.

Those within testing understand that use of automated testing tools isn’t a magic bullet. The language used gives the impression that an automated procedure is an easy procedure. It’s an understandable reaction. There are many fields in which automating procedures have made things very easy. However, the same thing isn’t true within our industry.

Using automation tools doesn’t make things easier, it’s just a different kind of difficult.

Now let’s think about what would happen if the term ‘automated testing’ was never used. The manager wouldn’t have in mind the magic automation robot finding every bug. The picture in mind would be similar to any craftsperson using their tools.

The language we use has repercussions in many ways and for a species which uses language as our primary means of communication; it isn’t something we can easily separate from anything else we do. It is inherent.

The way we talk about testing is part of our testing.

Don’t worry, it’s only minor – Bug severity in Oculus Rift testing

Bug severity always raises different opinions. We’ve all submitted a bug and seen it edited down to a lower severity. Severity ratings become a loose guide to the nature of a bug. They can be useful, but seeing the 1-5 rating does not relate enough information on its own.

x8cdw

There are bugs that are a lower priority to fix, but there are no minor bugs when testing in VR.

*ANY* bug can break immersion.

Our aim is to give users the most immersive and seamless experience possible.

A bug may be minor in nature, but its knock-on effects are never minor. A user may recover immersion quicker from a less critical bug but that does not make it minor.

Immersion is totally possible, but only if we make it the smooth experience it needs to be.

 

First thoughts – Testing with Oculus Rift

When I put on the headset for the first time; the immediate brightness instantly triggered my ‘design alarm’. Bad contrast and overly bright interfaces are one of my bugbears. It became apparent that it was going to be even more of an issue inside the headset.

It may seem obvious that overly bright interfaces would be worse in VR, but if it’s that obvious, why does it still happen on websites?

I noticed that line weights were dramatically reduced when viewed in Oculus Rift, rather than a monitor. This issue does connect to contrast. If your copy is rendering much thinner in a headset; it’s going to be very difficult to contrast between the copy and the surround.

I’ve been known to fuss a lot about contrast issues, but that’s because I believe it’s very important.

I fully support these people.

There are huge numbers of people with sight problems; both diagnosed and undiagnosed. If you present content that requires concerted effort of the user to read it; then you are isolating a big percentage of your possible audience.

Now extend this idea to VR.

If you create a product that alienates a fair percentage of your audience; they don’t decide to use another VR system. You’re not running a website where you may lose them to a competitor.

Alienating someone means they will most likely be lost to the world of VR. When you’re trying to present ‘the next big thing’ you need each and every person to go ‘WOW’.

If you make one person ‘WOW’ then they tell others, obviously the converse is true. You don’t lose one person to VR if they have a bad experience, you potentially lose more.

Good testing isn’t simply about pointing out issues with a user’s experience. It’s easy to say something has bad contrast and could be hard to read. It’s harder to see the knock-on effects that issue can cause. That’s where good testing comes in. The ability to see the problem and *potential* problems created by it.

 

Going freelance

So no-one ever tells you all the stuff you have to do at the start when going freelance!

You always hear the cool stories and how someone’s life/work balance is much better now etc… etc…

So it’s quite a lot to take in but I’m making the move, so if anyone does want to hire me then get in touch and we’ll see how I can help!

Check the about tab for contact details.

From Emile Durkheim to Max Weber – When context became key

In this article I want to talk about some integral figures within Sociology. People who helped to shape the discipline.

When studying Sociology and traversing the classic studies; Suicide by Emile Durkheim is usually the first studied. His aim was to study suicide across Europe. At the time no sociologist had attempted something on this scale. In his own words from the preface,

The progress of a science is proven by the progress toward solution of the problems it treats. It is said to be advancing when laws hitherto unknown are discovered, or when at least new facts are acquired modifying the formulation of these problems even though not furnishing a final solution. Unfortunately, there is good reason why sociology does not appear in this light, and this is because the problems it proposes are not usually clear-cut. It is still in the stage of system-building and philosophical syntheses. Instead of attempting to cast light on a limited portion of the social field, it prefers brilliant generalities reflecting all sorts of questions to definite treatment of anyone. Such a method may indeed momentarily satisfy public curiosity by offering it so-called illumination on all sorts of subjects, but it can achieve nothing objective. Brief studies and hasty intuitions are not enough for the discovery of the laws of so complex a reality.
(Durkheim, 1897)

Here we can see his laudable aims, but in the tail of the quote we see an issue. Durkheim supposes reality a complex place, but still thinks there are laws which can be discovered by Sociology. He is imagining being able to apply formulae like a physicist and gain the answers to our world. However, he also recognises how the word suicide may not always mean the same thing,

Our first task then must be to determine the order of facts to be studied under the name of suicides. Accordingly, we must inquire whether, among the different varieties of death, some have common qualities objective enough to be recognizable by all honest observers, specific enough not to be found elsewhere and also sufficiently kin to those commonly called suicides for us to retain the same term without breaking with common usage. If such are found, we shall combine under that name absolutely all the facts presenting these distinctive characteristics, regardless of whether the resulting class fails to include all cases ordinarily included under the name or includes others usually otherwise classified.
Durkheim questions how an act is defined and the agency of the person committing the act,
…in general, an act cannot be defined by the end sought by the actor, for an identical system of behaviour may be adjustable to too many different ends without altering its nature. Indeed, if the intention of self-destruction alone constituted suicide, the name suicide could not be given to facts which, despite apparent differences, are fundamentally identical with those always called suicide and which could not be otherwise described without discarding the term. The soldier facing certain death to save his regiment does not wish to die, and yet is he not as much the author of his own death as the manufacturer or merchant who kills himself to avoid bankruptcy?
Durkheim used whatever official sources he could to acquire the data. He then went onto present it in tables such as these,

 

suicideRates
and decides that,
The suicide-rate is therefore a factual order, unified and definite, as is shown by both its permanence and its variability. For this permanence would be inexplicable if it were not the result of a group of distinct characteristics, solidarity one with another, and simultaneously effective in spite of different attendant circumstances; and this variability proves the concrete and individual quality of these same characteristics, since they vary with the individual character of society itself.
Now whilst Durkheim has questioned the nature of the word suicide. He takes the collected statistics as fact. He sees the variance in statistics reflecting the attitudes of the society. What he doesn’t do is ask whether the attitude of the country initially decides when a death is a suicide.

If someone in a Catholic country found their friend dead with a suicide note; would they report that as suicide? Would they doom their friend’s body to burial in unconsecrated ground? How many suicides were counted as accidental deaths to give Italy the least amount of “suicides” in the above table? How many still happen today?

We can see that Durkheim is certainly questioning and exploring the nature of reality, but has blind spots which give Suicide its flaws.

If we move forward to Max Weber, the founder of the Interpretive school of thought. Weber introduced the idea of ‘Verstehen’
Verstehen is a German term that means to understand, perceive, know, and comprehend the nature and significance of a phenomenon. To grasp or comprehend the meaning intended or expressed by another. Weber used the term to refer to the social scientist’s attempt to understand both the intention and the context of human action.
(Elwell, 2005)
Weber decides that all societies have their own ‘norms’ and ‘values’. How something such as suicide was seen; would be determined by those norms and values.

From our earlier example; Italy’s norms and values see suicide as a mortal sin. These norms and values will therefore determine not only an individual’s likelihood to commit suicide, but also how likely that suicide will be recorded as such.

No respected sociologist today would ever attempt to analyse statistics without evaluating how the statistics were collected. Every sociologist today recognises that context is key to every investigation, even in terms of why someone decides to research a certain topic!

Context-driven sociology doesn’t exist because it doesn’t need to. How long until testing is the same?

So I was a sociologist…

Scrub that, I am a sociologist.

Being a sociologist doesn’t require a degree or a paid job title; it requires critical thought of the world at large. I did this before I ever gained the language to express it.

Taking an A-level course at age 16 in Sociology was one of the best decisions I ever made.

My teacher John Whincup was incredible in his ability to relate ideas and spur thought. He was the embodiment of everything you wanted in a teacher. He was able to entertain, engage and inform in an effortlessly charismatic fashion. I realised the thought-processes he was encouraging were how my mind already worked; I just needed the vocabulary. He mentored me and I’m grateful for the fact we still keep in contact fifteen years on.

After this period I went onto become a Computer Science and Sociology degree student and after a few years I became a Comp Sci degree dropout and eventually gained my B.A in Sociology. I then spent quite a few years having a fun and enriching time; although most people would probably call it being a loser.

Somehow I ended up working in Social Policy for a well-known British charity but due to various factors I couldn’t continue there. Through another serendipitous turn of events in November 2012 I ended up in the world of software testing.

I instantly felt at home.

After becoming acclimatised in my workplace I began to thirst for more knowledge. I wanted to understand the wider world of testing. I discovered the context-driven school of testing, James Bach and Michael Bolton who relate Cem Kaner’s idea that testing is a social science.

Now that got my attention.

Software testing is a social science.

I mulled it over and continue to do so. The idea that the digital software is just a vehicle, a human wrote it and a human is testing it. We have a collision of ideas; the interfacing of humans over the software is inherently social. Apparently requirements always suck which when thinking about it isn’t surprising. We humans can often suck at relating ideas to others, especially when we don’t know the person. Our interactions are littered with misunderstandings and that’s with people in our immediate life who may know us very well.

So as testers, we aren’t just testing software we’re also trying to understand the thoughts of the people who created it.

Another thing that rung bells with my sociology education was James Bach’s calculator exercise.

Now compare that to this classic sociology exercise which was one of the first things I remember John Whincup teaching.

There is a room with two people and a candle is lit, why?

roomCandleSociologyExercise

Hopefully there are a number of things going through your head.

There is no definitive answer to this question. This question is designed to ignite thoughts. It should make you realise that a scenario can have many different reasons for occurring. The question doesn’t teach you sociology, it teaches you how to think critically which is the entire basis of sociology.

The calculator exercise works in a similar way, it makes you ask yourself the questions that get your brain to work critically.

This blog will continue to record my attempt to apply what I’ve learnt from sociology into the world of testing.