EPUB Testing
[Narrator:] The structure of EPUB files is precisely defined. Their metadata define a clear navigation hierarchy. The supporting files need to be stored and referenced as defined by the standard. The packaging of the resources needs to be done following a strict procedure. Otherwise, you risk your publication being unreadable by any e-book reader. No matter how careful you are or what tool you are using to create a book in EPUB format, a few mistakes can creep in. That is why you need to validate your e-book before distributing it.
Validation is the process of making sure that your EPUB contains all the elements that e-book readers expect. An e-book reader can usually open an EPUB file that has not been validated; however, the contents might not render properly and the navigation might not work.
The grade of accessibility is defined by the EPUB Accessibility specification of the World Wide Web Consortium. As there are clearly defined rules, we can use software tools to test our documents. Tools can help us review our code for correctness. Does it contain any syntax errors? Do all our images have alternative text? Do we have the required metadata?
Don’t forget that even though tools can be of great help, they have their limitations. The software cannot determine the meaning of code elements. We need a practical test to check whether a publication can be used and understood by a person with disabilities.
EPUBCheck
EPUBCheck is a tool to validate the conformance of e-books to the EPUB specifications. EPUBCheck can be run as a stand-alone command-line tool or used as a library. It is open-source software, maintained by the DAISY Consortium on behalf of the World Wide Web Consortium. You can find the link to it in the Documents tab of the Accessibility site.
If you don’t like working with a command-line interface and prefer a more graphical user interface instead, then you could use the pagina EPUB-Checker. This tool incorporates EPUBCheck and offers some additional functions. It is available for free and runs on Windows, macOS and Linux. You can find the link to it in the Documents tab of the Accessibility site. The command-line version of EPUBCheck can be recommended when you need to embed it in an automated production workflow. The pagina EPUB-Checker, with its graphical user interface, is nice when working manually.
Let’s try to check a publication. We select an EPUB file and open it using the pagina EPUB-Checker for validation. The results of the validation will be shown in the window. The tool issues error messages and warnings. It shows the type of error, the path of the analysed file in the package and the position of the code in question in the file. EPUBCheck is great at finding problems. But in many cases, it’s lacking when it comes to explaining what those problems are.
EPUBCheck assumes that you have a certain level of knowledge and the know-how to fix the problem. In our case, the program is reporting a set of fatal errors caused by an unmatched paragraph element. The error message specifies the file and the line number. Let’s fix it. Every paragraph start tag requires a matching paragraph end tag. As we can see here, the paragraph was not properly closed. So we add the paragraph end tag. We save the file and restart the validation. Another error is reporting that the NCX identifier does not match the OPF identifier. If we compare these two values, we can see that their content doesn’t match. Let’s fix this too. We restart the validator and… Voilà! It now looks much better.
EPUBCheck is an excellent solution when searching for generic code problems, but it is a poor solution when it comes to finding accessibility issues. There are better tools for this.
Ace by DAISY
Ace by DAISY is a free, open-source tool for checking the accessibility of EPUBs. It was created to help evaluate their conformance to the EPUB accessibility specification. It performs a variety of automated checks, producing a list of violations that need fixing and data visualisations to aid the manual inspection process. It can be used as a stand-alone tool or can be integrated as a third-party component into broader publishing workflows or authoring processes. You can find the link to it in the Documents tab of the Accessibility site.
We start the app version of the Ace program. We open the same EPUB file that we have used before, containing the same errors, and start the validation. A report opens showing the validation summary in the form of a table. Several tabs give us access to the details. Under Violations, we see a table of the automated tests that have discovered problems, with suggestions for remedying them and links to the relevant sections of the DAISY Accessible Publishing Knowledge Base.
The table ranks the violations according to their level of severity: critical, serious, moderate or minor. It classifies violations according to their ruleset categories: Web Content Accessibility Guidelines 2 A, Web Content Accessibility Guidelines 2 AA, EPUB, Best Practice or Other. This is useful for getting a quick overview of the violations. The Metadata tab presents a table of all the publication’s metadata metadata from its package document, and makes special mention of present or missing accessibility metadata.
The Outlines tab presents extracted views of different aspects of the publication’s structure, presented side by side for quick comparison. The outlines are intended to assist with manual checking. They are: the table of contents, taken from the EPUB navigation document; headings, including indications of missing heading levels; for example, going from heading 1 to heading 3 with no heading 2 in between; and HTML, which is a view of just the heading content and sectioning content.
The Images tab presents a table of all the images in the publication and the accessible properties for each one. The table columns are: image, showing the image itself; the contents of the alt attribute; the contents of the element referenced by the aria-describedby attribute; the figcaption tag for the image; the location of the image in the EPUB; and the image’s role attribute value.
Let’s see how we can improve this publication. We’ll start with the violations. There seem to be some metadata missing, so we add them to the metadata block in the OPF file. Now let’s rerun the document check. All the validation errors related to metadata have been solved. Next, we have a serious error because the language was not defined in an HTML element. So we add it to the corresponding file and do another recheck. Again the validation returns fewer errors. The validator has detected insufficient colour contrasts for the headings in Chapter 5. We have to fix this in the style sheet and recheck. The validator is reporting a case of inappropriate heading-level nesting. We can see this problem in the outlines section. The file for Chapter 2 has a heading of level 3 under a heading of level 1. Of course, this is not correctly nested. We fix this so that we have a heading of level 2 following the heading of level 1. We restart the validator and now we can see that the outlines section no longer shows any problems. All violations have been resolved too.
Let’s look at the metadata section. We fixed our metadata problems. So all the metadata reported as missing here are optional. This is just a reminder in case we want to consider adding them. Let’s go to the images. We see all the images and their describing attributes. The second image has no alternative text. It shows a set of sparkles. This image doesn’t add any content to the book: it’s for illustration purposes only. When we look at the code, we see an alt tag attribute without any content. We can guess that the author intended to signal a screen reader to ignore this image. To increase the EPUB’s compatibility with as many e-book readers as possible, we add the role Presentation to this image. This tells a screen reader that the alternative description was intentionally left empty and the image should be ignored.
When we rerun our validation, we can see that the role has been added to the image. The alternative text for the third image just reads “Image”, it doesn’t describe it. We open the file in which the image is referenced and we update the alternative description to match the content of the image. Looking the alternative text of the next image, we can see that it starts with the text “Image:”. Please note that this is not a good style. A screen reader will present this information as an image, so this will present as repetition. We use our editor to delete this text fragment.
As the program cannot evaluate the meaning of alternative image descriptions, we manually check each one to make sure it matches the content of the image. This is the final step of our validation process. That’s it. Our EPUB passes all of the Ace program’s validation tests.
Practical tests
The previous tests analysed the code syntax and checked for misplaced or missing elements. These checks follow strict rules and can therefore be automated using software tools. Even though these tools can be of great help, they have their limits. The software cannot determine the meaning of code elements. We need a practical test to check whether an EPUB can be used and understood by a person with disabilities.
- Are the alternative descriptions correct?
- Do they describe the content of the image in such a way that a person with visual disabilities can get an idea of what they portray?
- Does the sequence of the content make sense?
- Is the colour contrast always sufficient?
Knowledgeable human evaluation is always required to determine if an EPUB is accessible. E-Books can be read on hundreds of different types of reading software and devices. It is totally unrealistic to test an EPUB on them all. As the validation procedure has made sure that our book can be read on as many devices as possible, it is sufficient to test the content aspects of three types of e-book readers.
- By testing on a PC, we can check whether the document works well with screen reader technology.
- By testing on a smartphone or tablet, we can check whether the document works with touchscreen technology. For example, are potential links in the document large enough to be usable?
- By testing on an e-book device with a greyscale screen, we can check whether the document can be read without colour.
For our example, we will use Apple Books on macOS in combination with the VoiceOver screen reader. We load our book and start the screen reader. Let’s try to read some text.
[Screen reader:] Heading level 1, Alice’s Adventures in Wonderland. Heading level 2, Introduction. New line. Alice with the Mad Hatter, March Hare and Dormouse at the tea party table, image. Heading level 3. This is a list of chapter titles.
[Narrator:] As you can see, VoiceOver reports the heading as a heading and presents the image with its alternative description. We come to a list.
[Screen reader:] Bullet, Down the Rabbit-Hole. Bullet, The Pool of Tears. Bullet, A Caucus-Race and a Long Tale. Heading level 3, Main characters. A little test table. Narrator: We come to a table.
[Screen reader:] Name. Description. Alice. Alice is a child in the mid-Victorian era. The White Rabbit. The White Rabbit appears at the very beginning of the book, wearing a waistcoat. The Hatter. The Hatter explains to Alice that he and the March Hare are always having tea.
[Narrator:] The VoiceOver screen reader reads the contents of the list and the table, but does not offer any information about their structure. Compared to the behaviour of a web browser, this e-book reader offers us much less information. Even though EPUB is based on web technology, e-book readers may be technologically lagging behind.
[Screen reader:] Alice’s Adventures in Wonderland, commonly shortened to Alice in Wonderland, is an 1865…
[Narrator:] Here is our book on an Android device. We are using the Google Play Books app and have uploaded the document to our device. We start the TalkBack screen reader.
[Screen reader:] Alice’s Adventures in Wonderland. Page 5 of 109. Showing full-screen view. Global Context Menu. Read from the top, in list 9 items. Double tap to activate. Alice’s Ad… Alice’s Adventures in Wonderland, page content web view. Alice’s Adventures in Wonderland, heading 1. Introduction, heading 2. Alice with the Mad Hatter, March Hare and Dormouse at the tea party table, graphic. This is a list of chapter titles, heading 3. In list 3 items. Down the Rabbit-Hole. Bullet, The Pool of Tears. Bullet, A Caucus-Race and a Long Tale. Test table with main characters, heading 3, out of the list. Name, column header, row 1, entry 4 rows, 2 columns. Description, column header. Alice, row 2, Name. Alice is a child in the mid-Victorian era, Description. The White Rabbit, row 3, Name. The White Rabbit appears at the very beginning of the book, wearing a waistcoat, Description. The Hatter, row 4, Name. The Hatter explains to Alice that he and the March Hare are always having tea, Description. Not checked bookmark. Double tap to toggle.
[Narrator:] When we start to read it, we can observe that Google’s book reader offers a better reading experience than Apple Books on macOS does: headings and alternative text of images get read to the user, and list and table structures are identified and presented as such.
[Screen reader:] Alert, Local Context Menu, not checked bookmark, double… Actions, in the list, 2 items. Double tap to activate. Alert, Actions. Next page, in the list 3 items. Alice’s Adventures in Wonderland. Page 5 of 109. Alert, Global Context Menu. Read from the top, in list 9 items. Alice’s Ad… Alice’s Adventures in Wonderland, page content web view. Alice’s Adventures in Wonderland, commonly shortened to Alice in Wonderland, is an 1865 novel written by…
[Narrator:] How is the book presented on an e-book device? This is what the book looks like on an e-book device that uses an e-Ink display. Currently, we are using the version without the colour corrections we made earlier. We are looking at the first page in chapter 5. Compare the heading with the one in the following chapter. Can you see that the contrast for the heading of chapter 5 is not sufficient? We’ve tested our e-book on a PC, a touchscreen device and an e-book device.
The reading experience was a little different each time. The validation procedure gives us a guarantee that we can read the information on all types of e-book readers, independently from the use of assistive technology.
Where to continue?
You have seen an introduction on how to validate and test an existing e-book that uses the EPUB file format.
Depending on your personal interests you could continue with the following chapter:
- EPUBs Adobe InDesign
[Automated voice:] Accessibility. For more information visit: op.europa.eu/web/accessibility.