Accessibility Review Strategies

Testing Approaches

Use automated testing first and then functional testing as needed.

Automated Testing and Functional Testing

Automated testing tools can identify some accessibility issues in your websites, web applications, or digital documents. Automated tools include:

• Browser add ons that identify issues on single webpages
• Cloud services that scan websites and provide reports
• Built-in checkers in document editing software

Automated tools cannot find 100% of issues (usually only 40-80%) and require human interpretation of results.

Functional testing, also called manual testing, can identify more issues that cannot be found with automated tools. This means actually reviewing digital content, trying to use it, and recording issues or barriers.

Automated testing, before manual testing, will:

• Identify a large proportion of problems
• Quickly give you an idea of the accessibility of your IT
• Narrow down the number of manual functional tests you need to perform afterwards

Testing by Workflow or View

Digital resources like websites or applications can have many interrelated parts. To decide which parts to test, you can select by workflow or view. A workflow is a series of steps a user needs to complete, while a view is a single section that a user encounters all at once.

Testing by workflow, also known as by test-case, can identify the most important areas of the website or application, in order to address those first. This approach can also find inconsistencies between views as you follow a workflow. This approach requires you to first identify the most common or essential user workflows.

Testing by view means checking each view as a thing in itself, without reference to others. This approach is easier to start, but can miss “whole picture” issues or issues that have an impact on critical functionality.

Which testing approach is best for your needs? The best approach generally depends on what you are testing:

• Informational or static IT resources (simple webpages, digital documents), test by view
• Interactive IT resources that require user input (web applications, forms or surveys), test by workflow

Automated Testing for Different IT

Static Website

Decide what to test:

• What pages are the most important?
• What pages are the most popular?
• Is this a CMS or template driven site?

Run automated tests:

• Use axe DevTools or another recommended browser plug in on each page you want to review
  • Export the report data for each page
  • Note which issues are coming from the CMS framework
• If the site is a departmental or unit site, or has a lot of internal or external traffic, consider using the Accessibility Scanning with axe Monitor service

Web Application

Decide what to test:

• What user workflows are critical?
• What functionality is the most frequently used?
• Develop test cases base on your answers

Run automated tests:

• Using a browser plug in, go through the test cases, exporting each report as you go along
• At the end of each test case, combine the reports into a single spreadsheet to represent the barriers for a workflow

Mobile Application

Decide what to test:

• What user workflows are critical?
• What functionality is the most frequently used?

Run automated tests:

• No available automated testing, proceed to functional testing

eBook

Decide what to test:

• Choose content with potential accessibility challenges to assess a publisher or vendor’s output
  • Ask for examples relating to STEM
  • Ask for books that may contain media, especially infographics, and/or formulas

Run automated tests:

• Check EPUB with EPUB-Checker and Ace

Documents

Decide what to test:

• What documents are the most important to the operations of your unit?
  • Documents distributed to students are important
  • Documents used repeatedly are important

Run automated tests:

• Use the accessibility checking tools available in the applications that created them

Functional Testing for Different IT

Manual functional testing is necessary because automated testing cannot find all issues. 

If you have done automated testing first, the effort and time for functional testing will be much lower.

You may choose to do a quick non technical review or a full review, depending on your needs. The first will help you flag problematic IT and may be useful as a quick test of vended products. The second will provide you results needed to fully address the compliance of existing IT