HTML testing

[Narrator:] In the previous chapters we have seen how to create accessible web pages. Once we have developed our website, how can we be sure that it is really accessible? In this chapter we will learn how to test an existing website for accessibility.

When should you start testing? Do not wait until your website is finished. Accessibility testing should be part of your development process. You want to identify problems early, when it is easier to address them. If an accessibility problem is part of your architecture, caused by a central library used for code generation, then it is important to fix this as early as possible. Otherwise, you may have to make major program changes and rewrites. Our advice: test as early as possible.

An accessibility evaluation should have two phases:

• The theoretical analysis.
• The practical tests.

The theoretical analysis

During the theoretical analysis, you will review the code for correctness. Does your code contain any syntax errors? Do all your images have an alternative text? Does the code use correctly nested headings? This analysis can be done manually or with the help of software tools. The theoretical analysis is based on the success criteria of the WCAG and ARIA standards.

Before we start our tests, we have to define our level of compliance, so do we test for A, AA or AAA compliance? The choice of level depends on the intended target audience. If you are creating a website for public use, then AA is a good choice. Most public tenders in the European Union ask for WCAG with AA compliance.

If your website creates dynamic web pages enhanced by JavaScript libraries to create a desktop like user interface experience, then ARIA is an additional requirement.

Once you have defined your level of compliance, then you know what to test for. The standards offer a precise checklist. Of course, you are free to check your HTML code manually for all the success criteria, but this can be a tedious task. Who wants to check thousands of images for alternative texts? This is where software tools come in. They can carry out hundreds of checks in a few seconds.

The Web Consortium maintains a list of Web Accessibility Evaluation Tools. You will find the link under the Documents tab on the platform. At the time of creating this course, the list contained more than 100 tools. Some offer very limited functionality, for example only checking colour contrast. Others try to test every possible WCAG success criteria. Some can be installed and executed locally whereas others require online services. Some focus on local regulations, for example the Section 508, US federal procurement standard, whereas others are dedicated to WCAG. Some require a commercial license, whereas others are free to use or even open source.

The list is a good start to find useful tools, but please note that this list does not claim to be complete. There might be other tools.

We will now show you how to use one of these tools. Note that we are not endorsing this tool over others. We have chosen this tool because it offers a lot of functionality and is available cross platform for free.

Although software tools can be used to automate checking for accessibility criteria, they have their limits. No tool alone can determine if a site meets accessibility standards. Knowledgeable human evaluation is always required to determine if a site is accessible.

The Web Developer extension: The Web Developer extension is a free tool. It adds various web developer tools to a browser. The extension is available for Chrome, Firefox and Opera, and will run on any platform that these browsers support, including Windows, macOS and Linux. Although this extension does not primarily focus on accessibility, it offers a lot of functions to support the creation of more accessible web pages.

The extension can be installed and uninstalled using the built-in functionality provided by the browsers. For our demonstration, we will use the Firefox browser on macOS.

The extension has been installed and can be reached via a little cogwheel icon located next to the address bar. We will not explain all the functions of this tool, but highlight only those which help us to improve the accessibility of our web pages.

Here is an example web page, which we will use to demonstrate how to use the tool. In the CSS menu, we find the function Disable all styles. We can now see the pure content of the page without any styling. In fact, this feature shows us the elements of the web page in the code sequence used by a screen reader. Does the content presented in this sequence make sense? If not, we should consider rearranging the code.

We continue with the Images menu. When activating the function Display alt attributes, the tool will display the available alternative text attribute values close to the images. Every image with content should have a describing alternative text next to it. If we do not see any alternative text, then no meaningful text will be read to users with visual disabilities. We can go even further and activate the function Replace images with alt attributes. This now looks closer to what a user with visual disabilities gets.

The function Outline images without alt attributes helps us to find all the images without alternative texts.

We continue with the Information menu. We can use the function Display title attributes to see all the title attribute values, such as the one describing the table content. If an image has any title information, the title value will be displayed here.

We select Display ARIA roles. Here we see the roles that we defined explicitly. Please note that implicit ARIA roles set by an HTML element are not displayed, for example the navigation role for the HTML nav element, or the main content role for the HTML main element.

If your web page uses access keys, then we can see them via Display access keys. This page defines two access keys. If we want to check for exceptions of the tab based keyboard navigation sequence, we can activate the Display tab index menu.

The View document outline function shows us the structure of the document created by the headings. In our example we see a correctly built document structure. Here is an example of how an incomplete structure would look. You can see immediately that a heading level 4 follows a heading level 2, which is not the correct way to nest headings.

Under Forms, we find the function Outline form fields without labels, which helps us to find form input fields without labels.

We continue with the Outline menu, and select Outline headings. We can see all the texts that have been marked as headings. This function is useful to check if we missed a text that should be marked as a heading.

All the functions we have seen so far can be used offline. The result gets more expressive if we use the online services.

In the Tools section, we find all the tools that use external online services to extend their functionality. We select Validate HTML. This function forwards the URL to the W3C Markup Validation Service. This tool checks for errors in the HTML syntax. We can see the results of the validation marked with according priorities. We will explain this online tool in more detail in a few seconds.

Next we activate Validate accessibility. This function forwards the URL to the WAVE accessibility evaluation server. A new web page opens presenting the result analysed by WAVE. We can see a summary of the evaluation on the left side and the annotated web page content on the right side. We will explain this online tool in more detail in a few seconds.

All in all, the Web Developer extension is very helpful to find potential accessibility problems. However, it does not interpret or prioritise potential problems. You have to know what you are searching for. It is more a debugger than a validator.

The W3C Markup Validation Service: The W3C Markup Validation Service is a free service by the Web Consortium that helps to check the validity of web documents.

Most web documents are written using a markup language such as HTML. These languages are defined by technical specifications, which usually include a machine-readable formal grammar and vocabulary. The act of checking a document against these constraints is called validation, and this is what the Markup Validation Service does.

Validating web documents is an important step that can dramatically help improve and ensure their quality, and it can save a lot of time.

Although the validation service does not primarily focus on accessibility, it can be very useful to find damaged HTML code. Damaged HTML code can confuse assistive technologies like screen readers, causing them to read items in the wrong order or not at all. As we have previously seen, the validation service can be invoked via another tool or directly from the web page. When using the web interface, you can simply enter the URL of the web page to get an evaluation.

If the website is behind a firewall or the code is only available locally, then you can upload or copy-paste the HTML code using the file upload or direct input methods.

In our example, the validation service reports that alternative text attribute values are missing for images. As these are a requirement of HTML 5, this is marked as an error. In another place, the validation service notices that tags have not been closed properly.

The validation service differentiates between different priorities of messages, so there are errors and warnings. Here we see a warning that an ARIA role definition is not necessary. We have declared the listitem role in list item tags, which implicitly have this role. So it does not do any harm, but it is not elegant coding either. Therefore, we only get a warning and not an error.

Let's fix these problems and then re-check. The validation service now confirms that our code does not contain any syntax errors.

The W3C Markup Validation Service is a very useful tool and its use is highly recommended.

The Web Accessibility Evaluation Tool, WAVE for short: WAVE is an online service for evaluating the accessibility of web pages. WAVE is developed and made available as a free community service by WebAIM. Originally launched in 2001, WAVE has been used to evaluate the accessibility of millions of web pages.

WAVE can be used via its web interface or via other tools, such as the Web Developer toolbar. It offers an extension for Chrome and Firefox browsers. An advantage of the browser extensions is that they work without connecting to its online service. The evaluation is done directly in the browser and no information is sent to the WAVE servers. This ensures that the accessibility reporting is 100% private and secure. This way it can be used in intranets, behind firewalls or with code stored on a local hard disk.

Let's use the online version of WAVE to evaluate our web page. On the left side of the screen we see a summary of the results. We have two errors and a few alerts. Features, Structural Elements and ARIA include all elements that improve accessibility. Contrast Errors indicate colour problems.

Next to the summary icon, we see 4 more icons. Details lists all the WAVE icons in the page, including their description. The icons are sorted by priority. Red indicates an error and orange an alert. Icons using other colours are for information purposes. If we click on one of them, WAVE will guide us to the position where the information is located on the page.

Every short description is followed by an info icon. If we click the info icon, the system guides us to the documentation. The documentation offers an extensive explanation of the WAVE icons and how to use this information to make your page more accessible. By clicking the Icon index link, we get an overview of all existing icons. Selecting an icon leads us to the documentation. The fourth menu icon shows the outline of the document structure. If there are any problems, WAVE will indicate this with an alert icon. In our example, a heading of level 4 immediately follows a heading of level 2, which breaks the heading navigation structure.

The fifth icon is dedicated to colour contrast. The last item shows any automatically detectable contrast issues. In our case, the links in the sub-menu do not have enough contrast. If you want to see the page without any colour information, press the Desaturate page link. If everything can be properly used and read without colours, then you can be sure that your page can be used by people with colour deficiencies.

We can deactivate all styles, which offers us a list view of the document using the sequence as it would be seen by a screen reader. We have read the result of the validation via the left summary bar. You might prefer to use the icons in the content part of the window.

Let's click on some of the icons to try to understand the accessibility problems. We click on the red label icon and a pop-up opens, which offers a short explanation. By clicking the Reference link, we are guided to the associated documentation. Feel free to experiment with it. Click the icons and try to gain an understanding of how your page works and how it can be improved.

On the bottom part we see a code link. Clicking it opens the code bar. When clicking icons in the page, the code editor will show the matching HTML code, to help you understand the problem.

In a previous chapter we compared two similar looking web pages. One of them was much less accessible than the other. One of the key problems of this page was the inability to detect navigation elements and navigation hierarchies properly. Let's test this web page with WAVE. There are five errors indicated by red icons and six alert messages. There are fewer blue icons, which indicate missing structural elements. There are no green or violet icons because this page does not contain any code that would offer an accessibility feature or create an ARIA role. So is this page properly accessible? As we have seen in our demonstration the answer is clearly No. The absence of errors does not mean your page is accessible or compliant. Although software tools can be a great help, they have their limits. They cannot determine the meaning of code elements. We need a practical test to check if a web page can be used and understood by a person with disabilities.

The practical tests

Although a tool like WAVE can help a lot, it cannot detect what kind of information is missing to make the page accessible. Software tools have limitations. They cannot: Figure out how to improve usability. Find missing document structures. Detect the meaning of elements. Check different types of colour contrast. So practical testing is not just an option. It is a requirement.

For successful practical testing define your test environment. As desktop platforms we recommend using the Apple macOS or the Microsoft Windows operating systems. For mobile use we recommend using an iOS or Android device. Each device should be running a recent version of the operating system. If you do not have multiple hardware and software solutions available, then test using the ones you have. We recommend cross-platform testing to ensure that your code works on any device and operating system version.

As time is limited, you cannot test every single aspect of every page of a website containing thousands of web pages. In this case, select the pages that are most representative. Content management systems often generate web pages from templates. Pick at least one page for every template you are using. Do not only test the web pages that offer the least content. Test the ones that are most complex and offer the most content. Test for all functions a template might implement.

When it comes to testing for the needs of visually disabled people, you need to learn how to use a screen reader. It is important to understand the basic working principles of a screen reader for successful testing. Please refer to the chapters describing how to use a screen reader. Create a to-do list of what to test and how to test it. This list will vary depending on the size and complexity of your website.

Here is an example of a to-do list:

• Identify common web pages of the website. For example, the home page, all types of content pages and all pages containing any type of user interaction.

   

• Identify the essential functionality of the website.

   

• Is the navigation intuitive and easy to use? Does the navigation match the architecture of the website?
• What kind of information does your target audience expect and search for? Is it easy to find?
• Identify the variety of web page types.
• For responsive websites: do the test results still apply for all resolutions and target devices?
• Does the website offer different functions for different user groups?
• Include processes. For highly interactive websites, discover typical processes and test them. What core functionalities need to be tested, such as online shopping?
• Check if the downloadable documents are accessible, for example PDFs or Microsoft Word files. A website is considered to be accessible when all its components are accessible, including downloadable documents.
• Test for different target audiences: test using a screen reader, test without colours, test without audio when offering audio/video content, test using the keyboard only.
• Document your results in a report: write an executive summary, what has been tested?, how has it been tested?, what tools and methodologies have been used?, what were the results?, what are the recommended actions?

For all tests, the success criteria of the WGAC and ARIA standards are the reference. Some of the questions cannot be answered by the developers.

The person who designed the architecture of the website cannot judge if it is easy to use. It is therefore important to involve external users in your tests. When testing it is often not easy to judge where usability ends and accessibility starts. Usability is the ease of use and learnability of a human-made object. In human-computer interaction and computer science, usability studies the elegance and clarity with which the interaction with a computer program or a website is designed.

Accessibility is the degree to which a product, device, service, or environment is available to as many people as possible. Accessibility is about making things accessible to all people, whether they have a disability or not. Testers should have a solid understanding of: web technologies, accessibility barriers that people with disabilities experience, assistive technologies and adaptive approaches that people with disabilities use, evaluation techniques, tools and methods to identify barriers for people with disabilities.

If you are not sure if you have all these competencies, you might want to consider asking a professional expert for help.

Last but not least, accessibility is a process, not a status. Websites change and evolve over time. Define regular intervals for tests, so that you can be sure that your website stays accessible.

Where to continue?

You have seen an introduction on how to test a website for accessibility. This is the final of the web standards chapters. You have been introduced to the WCAG and ARIA standards. You have seen what they are and how their application can improve the accessibility of web pages. Example HTML and JavaScript codes, which demonstrate how accessibility can be improved. How to test web pages for accessibility to ensure that they are really accessible.

Depending on your personal interests, you could continue with one of the following chapters. As testing requires extensive knowledge about concepts, you might want to revise one of the chapters on Concepts for a deeper understanding:

• Assistive technology
• Screen readers
• Fonts, sizes and colours
• Alternative text
• Text transcripts, captions, and sign language

If your website offers downloadable documents, you might want to make them accessible too. You can find more information about other file formats in one of the following chapters:

• Introduction to accessible PDF
• Introduction to accessible EPUB

[Automated voice:] Accessibility. For more information visit: op.europa.eu/en/web/accessibility.

Close tab