A Comprehensive Guide to Accessible User Research: Part 2 – Recruitment and Preparation

Including people with disabilities in user research studies reduces the risk of delivering a proposition that is not useful or accessible for people with disabilities. It’s a crucial and fundamental aspect of inclusive design. This article looks at the considerations for preparing accessible user research studies and recruiting for participants with access needs.

This article is the second of three:

• Article 1: Planning for accessible user research
• Article 2: Preparing and recruitment for accessible user research
• Article 3: Conducting and reporting accessible user research

If you haven’t read the first article on planning accessible user research, read that first. It’ll help you with the broader project considerations like goals, budget, or location of the research. This article is for readers who are ready to progress with accessible user research. It will focus on some of the specifics when it comes to preparing research and recruiting for a research study.

Kick-off your recruitment

Whether you are recruiting a proportion of people with disabilities into each round of your research plan or conducting dedicated accessibility research where all participants have a disability, there are two key considerations:

1. Ensuring that a variety of disabilities and assistive technology users are being represented
2. Ensuring the prototype is appropriate for the impairment and/or assistive technology

These two considerations are very intertwined depending on how you decide to run your research.

A thorough research study will ensure that participants with a variety of impairments have been included, like visual, physical, cognitive (including learning and neurological), auditory, and speech impairments. It will also ensure there is a variety of participants within one of those categories. That could be the severity of the impairment, the level of experience using assistive technologies, or the brand of assistive technologies.

For example, within visual impairments, you should aim to recruit participants who are color vision deficient, have low vision, or are blind. Of those participants who use screen readers, you should recruit people who use different brands of screen readers, like JAWs, NVDA, TalkBack and VoiceOver.

While it’s useful to have categories to use as a framework for recruitment, no person fits neatly into defined categories. It’s common for people to have multiple impairments so it’s important to focus on the access needs and how people use the Web in different ways rather than the impairment itself.

The fidelity of the prototype is another key aspect of recruitment consideration. If your prototype is a paper-prototype, wireframe prototype, or a prototype with limited functionality, then there will be limited value in recruiting people who use assistive technologies like screen readers, or speech-to-text software. That’s because assistive technology requires content to be programmatically available.

There are ways around this, for example a research moderator can mimic the speech of a screen reader in a wizard-of-oz style facilitation, but this requires a highly skilled and experienced accessibility researcher.

Here are some common examples to help us think about how we might recruit in the different stages of research:

Example 1 – Generative or discovery research

Context: contextual research at the participant’s home to understand how they go about their day-to-day tasks.

Recruitment consideration: at this stage of research, the recruitment should aim to find a variety of participants—people with different disabilities, people who use different assistive technologies, and/or people who use the Web in different ways.

A researcher might also prioritize people who are not mobile, and who may be difficult to represent in latter stages of research which might require travel, or the use of their own assistive technologies. The criteria doesn’t need to be overly specific.

For example, depending on the aims of the research, it may not be important to represent a cross-section of users who use different brands of screen readers, but instead to focus on how people with different severities of vision go about their day-to-day tasks.

Example 2 – Agile research during product iteration

Context: iterative user testing to understand how well the prototypes work to address a user problem.

Recruitment consideration: at this stage of research, the recruitment should aim to match participants to prototypes that are suitable for evaluation, while aiming to include people with a variety of access needs across sessions. What that means is you might not have a variety of access needs in each round of research, but you’ll have reasonable variety across the separate research studies.

The other consideration is which access you need to recruit for in each round. As a general rule, if the prototype is available in HTML and functional, then you can include participants who use assistive technologies. If a prototype isn’t functional, you shouldn’t recruit participants who use assistive technologies.

Example 3 – Evaluative research pre-launch

Context: lab-based user studies to understand whether products are accessible and usable to participants.

Recruitment consideration: at this stage of research, the recruitment should aim to find a variety of participants who use the Web in different ways, with a view to ensuring a product is accessible to as many people as possible at launch.

The main recruitment difference between this research and example 1, is that the recruitment here should be more specific and ensure that there is good representation across the spectrum of disabilities and assistive technologies.

For example, the recruitment spec might ask for 3 screen reader users (1 JAWs user, 1 VoiceOver user, and 1 NVDA user); 2 screen magnification users (1 Zoom, and 1 ZoomText), and so on.

If you’re able to consistently run research with people with disabilities, you’ll be putting your team and business in a great position to provide accessible products and services. But if you’re only able to include one participant with an access need in your research, that’s OK too. Accessibility is a journey, and isn’t solved in a day.

Finalize the location

In the first article, we discussed some of the merits of different research locations, and how the aims of the research and access to assistive technologies can influence the location.

If the research is taking place in the participant’s home or remotely, there are no big additional considerations beyond those discussed in the first article.

For a home-visit, you may want to bring additional recording equipment, if for example you want to record the participant with one camera, and the assistive technology with the other camera.

For remote research, you need to ensure that every aspect of the web conferencing software you’ll be using is accessible, including registration, settings/preferences, and screen sharing. Some software even allows for live captioning now, which will be of great benefit to many of your participants. If you can send instructions in advance and schedule a test call, you will reduce the risk of losing valuable time with the participant on the research day.

If the research is taking place in a lab, it’s important to visit the lab in-person. Ensure there is step-free access throughout including doorways, entrances, bathrooms, and emergency exits. Also make sure that there is enough physical space for a guide dog, for wheelchair access, and for an additional person like a care worker or interpreter.

The furnishings need to be suitable as well, such as flat desk surfaces, adjustable desk heights, and different options for chairs (swivel chair and stationary chairs). Soft rugs can reduce mobility for wheelchair users and can be a trip hazard for someone with one-sided muscle weakness. You may also have participants who prefer lights to be dimmed.

There can be a lot to consider which may be overwhelming for some. However, the most important thing to remember is that the space should be adaptable so that you can accommodate different preferences and needs.

Lab-based research also requires researchers to check that labs have access to the different assistive technologies your participants use. It would be unreasonable to ask a JAWs user, to use the lab’s NVDA or VoiceOver software—they aren’t identical screen reader software. You need to match the brand, and even try to match the version of the brand as best you can. If that’s not possible and your participant is willing to bring their own device, make sure you have the necessary recording equipment available to capture the data you need.

Prepare your materials

If you can finalize the recruitment and the lab location early, that will give you more time to send out materials to the participant in advance and in their preferred format. Some typical materials to prepare for user research include consent forms, information sheets, and incentive receipts. Common adaptations for these materials may include using large text and plain language.

Oftentimes, sending the test materials in advance can be enough accommodation. It gives participants time to read the materials in their own time. This removes any time pressure a participant might feel if they have to read and sign the consent form at the start of their research session.

Many people might assume that materials for people who are blind need to be prepared in braille. This is true for people who read braille, but not all people who are blind use braille. If your participant is using a screen reader, send your materials over in a digital format that is accessible to screen readers.

Keep in mind that PDFs can be inaccessible for some users, depending on how it’s been designed, written, and tagged. To reduce the chance of creating inaccessible PDFs, keep the layout really simple and use simple language. Better yet, avoid PDFs and send the materials directly in the email.

Other alternative formats include using online platforms, for example a survey tool to administer a consent form. You might also provide an information sheet as an audio clip, or a video with a talking head.

If you’re unsure which format you should offer to your participants, just ask the participant directly—the important thing to remember is that you should be prepared and have time to offer alternative formats, and the formats you use should be as accessible as possible.

Conduct pilot tests

In the lead-up to conducting research, it is typical to run a pilot test so that the researcher can get familiar with the discussion guide and iron out any kinks.

If possible, you should aim to over-recruit and have people with disabilities in the pilot tests. Conducting a pilot test with a colleague or user who doesn’t have access needs won’t prepare you in the same way.

Running pilot tests will ensure that your prototype doesn’t have any significant barriers prior to testing, or will at least identify known barriers so that you can advise the participant and facilitate the session appropriately. It will also ensure your sessions are well timed and give you an opportunity to get more familiar with assistive technologies. You should also check that recording equipment is capturing all the data you need to collect.

If your pilot runs smoothly, you should include the data you collect as part of your analysis.

In summary

This article covered specific considerations around recruitment, and various aspects for preparing the research study. The common theme across all of these topics is to give yourself additional time for adequate preparation.

The additional time will allow researchers to ensure that a variety of voices are being represented, the test setup, materials, and technology is adaptable, and that pilot tests can be conducted and protocols adjusted as necessary, before proceeding.

The next and final article of this series will go into more practical guidance for on the day of research, including setting up the lab, facilitation, and reporting.

This article is the second of three:

• Article 1: Planning for accessible user research
• Article 2: Preparing and recruitment for accessible user research
• Article 3: Conducting and reporting accessible user research