The Challenges Of Accessibility Testing

The Challenges Of Accessibility Testing

Accessibility testing is really important to engage with, to ensure that everyone who wants to play your game has a fun and fair experience. Studios are increasingly encouraged to take part in accessibility testing and accessibility-specific development, and while it is important, it does present its unique challenges.

We did a two-year accessibility life cycle project, constantly gathering feedback from a range of testers, to hold stakeholder meetings and prioritise the required development work in terms of impact, priority, and investment requirements.

Before I get into the challenges that we faced, here is some basic methodology.

We created a google form that captured basic information about our testers, their name, age, and an open description of any conditions they have and what that means for them, and what their challenges are. This also included any special devices which they typically use to game and what these are.

We then reviewed all the respondents, sorted them into Cohorts, and selected ten players from each. Our cohorts were;

• Those with visual impairments

• Those with cognitive impairments

• Those with motor impairments

• Those with auditory impairments

I then crafted a survey for each of the four cohorts to capture their feedback about the accessibility features, and anything that they struggled with or did not like, or even loved. All of these included an option to add any extra feedback they wanted to share that we had not previously asked about.

The game was then distributed to each group of testers via the Go Testify platform with a link to the applicable cohort survey.

Comorbidity

The very first challenge that I faced was Comorbidity. Comorbidity refers to “the simultaneous presence of two or more diseases or medical conditions in a patient”

The challenge that this presented for me was that some of our testers would not fit into just one group. For example, someone who has MS will suffer from motor and visual problems, while people who experience complete blindness often have other physical limitations.

This meant that the survey that people with comorbid conditions received did not encompass all of the information that they may want to share, and the feedback may be limited or misconstrued as a result.

Of course, the mediation we took for this was to add an open question at the end of the survey asking if there were any other issues or challenges or feedback they want to share.

Another challenge the studio faced was Adaptation.

As the feedback rolled in, players made us aware of their barriers and we began to see how we would fix them. However, what we found was that some solutions for certain players created bigger challenges for other players.

Take for example players who struggled with their motor abilities in their hands. Some of these motor conditions caused repeated accidental actions like clicks, and so players wanted a double click to confirm or a press and hold so that they would not accidentally skip the story. While other people with motor problems would struggle more with a press-and-hold or double-click interaction due to pain or inflammation in their hands.

This was feedback that the studio had to contend with in terms of what is the bigger consequence to the players in both instances. Who is most impacted and what proportion of people are reporting each problem?

Technical problems

Some of our users, particularly those who were partially or completely blind could not use our remote recorder. Our recorder relies on reading the information on the screen to download, locate, set up, and use the build with screen and voice capture.

We found that any software that players used like text readers were not compatible with our recorder. This meant we just needed a new way to allow these people to take part, the studio was able to supply the build to the tester directly via a secure email, a process the tester was comfortable with using and familiar with.

The enjoyment that the studio and I had from running and analysing this feedback was extremely rewarding and fun. I hope that by sharing the realities and challenges of accessibility testing studios are still encouraged to engage with it and that you can get stuck in preparing for the hurdles that you might face and be more equipped to handle them.