Evaluating Accessibility Compliance Documentation

Compliance documentation (CD) types

Important: If the purchase presents a high risk, the actual product will need to be examined and evaluated and the results compared to the documentation, helping us to score the vendor.
  1. Vendor questionnaire answers: In any RFP, the vendor is required to return answers to a questionnaire that is attached to the RFP. This is the only CD that is required.
  2. VPAT: Standing for “Voluntary Product Accessibility Template,” it is a self-disclosing document produced by the vendor (or a third party) which details each aspect of the Section 508 requirements and how the product supports each criteria.
  3. Other: Vendors may publish statements about their compliance on their website. While you are there, examine the website itself for accessibility using a quick non-technical test. If their promotional material is not accessible, the product may be deficient as well.

Vendor questionnaire answers

How to score the table below: A yes response to a question scores the value in the right column. A no response scores a value of 0. The maximum score is 30 points.

Question Answer Value
1
 Vendor provided accessibility information about the product
+2
2
 Vendor outlined how they account for accessibility in company policy and practices (including training, development, quality assurance)
+5
3
 Vendor or a reputable 3rd party tests with actual users with different disabilities
+5
4
 Vendor or a reputable 3rd party tests with a range of assistive technology
+5
5
 Vendor provided an accessibility contact in the company and this person is not in marketing
+3
6
 Vendor prioritizes fixing accessibility errors and does not treat them as feature requests
+3
7
 An appreciable percentage of development and QA is devoted to accessibility
+3
8
 Vendor says product is not accessible, but has provided a roadmap to make it so
+2
9
 Vendor provides details on training on accessibility for all roles involved
+2

VPAT analysis

How to score the table below: A Yes response to a question scores the value in the right column. A No response scores a value of 0. The maximum General Score is 15 points.

General Score

Question Answer Value
1
 VPAT is the most current version, and it is a VPAT - WCAG or a VPAT - INT
+5
2
 Vendor provides detail on how product was tested and who tested it
+5
3
 Vendor demonstrates that VPAT is relevant to current version of product or is otherwise up-to-date
+5

Success Criteria Score

Success Criteria score will depend on the number of criteria that are “Not Applicable” - apply the following formula for maximum score (15 + [(38-N) * 5] where N is the number of criteria that are not applicable. Once you have the maximum score you can calculate what the actual score is as a percentage of the maximum score.

For each success criteria

Question Answer Value
4
 If the conformance level is "Supports" and VPAT provides detailed remarks that explain why this conformance level was chosen
+5
5
 If the conformance level is "Partially Supports" or "Does Not Support" and  
5-A
 • VPAT provides detailed remarks about what features are non-conformant
+1
5-B
 • VPAT provides information about a remediation plan
+2
5-C
 • VPAT provides information on  a reasonable remediation timeline
+2

Provisionally, the VPAT needs to score 10 points of the General score and 90% of the criteria score to be acceptable. This method will be refined and communicated as we practice it in real life situations.