Automated testing first where possible
Why automated testing first: Using some software tools will help identify with little effort a large proportion of any problems your IT may have and will narrow down the number of manual functional tests you will need to perform via guided inspection and use of assistive technology.
What it does: Automated testing tools help you find accessibility issues in your information technology (IT), organizing them by priority and type.
What it does not: Automated testing tools will only identify about 40%-50% of the potential issues in your IT. It also does not focus on user experiences that people with disabilities may have. Manual functional testing is also required.
Automated testing for different IT
| Prioritize ➔ | Automated Review | |
|---|---|---|
| Static website | What pages are the most important? What pages are the most popular? Is this a CMS or template driven site? |
If the site is a departmental or unit site, or has a lot of internal or external traffic, consider using Deque axe Monitor. If not - use axe Expert or another recommended browser plug in on each page you want to review. Export the report data for each page. Note which issues are coming from the CMS framework. |
| Web app | What user workflows are critical? What functionality is the most frequently used? Answering these questions will provide you with test cases. |
Using axe Expert or another browser plug in, go through the test cases, exporting each report as you go along. At the end of each test case, combine the reports into a single spreadsheet. These would be some of the barriers someone may encounter for a particular workflow. |
| Mobile app | What user workflows are critical? What functionality is the most frequently used? |
There is no easy way to automate mobile testing. One possible shortcut is to test only in the most popular mobile devices in your context. |
| eBook | If you are examining a vendor’s compliance in general, ask to examine items relating to STEM. If not, ask for books that may contain media, especially infographics, and/or formulas. |
Check ePub with EPUB-Checker |
| Documents | What documents are the most important to the operations of your unit? Documents distributed to students are automatically important. | Use the accessibility checking tools available in the applications that created them. |
Functional testing for different IT
Manual functional testing is required, as automated testing can only do so much. But if you have done automated testing on your IT already, the effort and time spent on functional testing will be much lower.
You may choose to do a quick non technical review or a full review, depending on your needs. The first will help you flag problematic IT and may be useful as a quick test of vended products. The second will provide you results needed to fully address the compliance of existing IT.
| Heuristic “non-technical” Review ➔ | Full Review | |
|---|---|---|
| Web | Use the keyboard to navigate and operate and navigate. Examine the contrast of the view elements. Zoom in up to 200% |
Examines your IT against all of the Web Content Accessibility Guidelines 2.1 AA, the standard adopted by the university. Our guide focuses on testing only those things that automated tests cannot cover. |
| Mobile | Requires some setup. Turn on VoiceOver on iOS, or Talkback in Android, navigate around the app with a few gestures. As you navigate, use only the audio to make decisions and perform actions. Note where VoiceOver or Talkback are at variance with what you see visually. |
Examines your IT against all of the Web Content Accessibility Guidelines 2.1 AA, the standard adopted by the university. It requires a significant investment in learning and reviewing. |
| eBook | Manually spot check EPUB using DAISY Knowledge Base as reference. | Use SMART. See a walkthrough of a full process using EPUB Checker, Ace and SMART |
Test case driven testing and view driven testing
For web and mobile, another possibility is to do test case driven testing or view driven testing. Test case driven testing may be more appropriate for web applications, view driven testing for static sites. A combination of both may be best.
| How it works | Pros | Cons | |
|---|---|---|---|
| Using test cases |
Write test cases, e.g. “User will log in, fill out her profile but skip a required field and submit” Follow the instructions in the test case, noting issues |
Flags issues in important workflows Surfaces inconsistencies between views |
Need to write test cases May not result in complete coverage |
| View by view | Examine views in application, noting issues. | Fast |
May miss a blocker in a critical function Will not address equity of workflow (e.g. all views are ok, but the aggregate imposes a burden) |