QA/Firefox3/TestPlan/UIAccessibility
Firefox 3 UI Accessibility Test Plan
Contents
Overview
This will outline the process to verify the accessibility of the Firefox 3 user interface. Tests will be designed to protect against regressions in the accessibility code, as well as to ensure the accessibility of new UI elements as they are added during the beta process. Testing will primarily be done through manual Litmus test cases, supplemented by object inspection and event testing using the rellevant Linux and Windows tools. Gecko rendering of websites (AJAX/ARIA, web forms, etc.) will be tested using dogfooding of nightly builds and specific test cases.
= Scope of planned testing =
The accessible UI will be tested with a combination of Litmus test cases using assistive technology products as well as testing tools such as Inspect32, Accevent and Accerciser. Due to the current differences between assistive technologies across platforms, automated testing would have limited effectiveness.
Platforms
This test plan will be carried out on all Win32 platforms, as well as 32-bit Linux platforms. Mac, 64-bit Windows and 64-bit Linux platforms will not be tested.
Platform Differences
Currently, the underlying accessibility interfaces between Linux and Windows are separate, but theoretically equal in functionality.
Every effort will be made to achieve equal functionality on the Windows and Linux platforms. Interface functionality (keyboard, focus logic, controls) should be identical.
Major Test Areas
Manual Testing
- Keyboard Navigation
- Test that the keyboard can be used to focus and interact with all UI elements that are actionable.
- Ensure that generally accepted cross-platform keyboard standards are used in all aspects of the interface (space bar to activate buttons, etc.)
- Ensure that all toolbar functions and other commonly used functions are accessible via single keystrokes
- Test that all UI elements give specific and usable information to the underlying accessible interface, depending on platform
- Label
- Grouping
- Role
- State
- Description
- Test that the caret can be accurately tracked in any Firefox input field
- Error and Notification Handling
- Ensure that errors interrupt the user experience and are communicated via the assistive technology
- Test that notifications are properly focused and communicated to the user by the assistive technology, in a similar manner to their visual presentation (e.g., if they do not disrupt the view of the webpage, they should not interrupt reading of a webpage via assistive technology)
- Test focus throughout the browser
- Test that focus flows logically from one browser element to the next when the tab key is used (location bar, search box, (toolbar?,) tab controls, main browser window
- Test that focus is brought back to a logical position when exiting menus/dialogs/closing child windows/switching focus back to the Firefox application window
- Ensure that focus is never null
- Test that cursor keys can be used to traverse the entire browser window
- Test that focus can be followed by magnifiers
Object Accessibility Testing
- Objects using MSAA/Iaccessible2/AT-SPI/ATK controls
- Label
- Role
- Accurate state information
- Logical grouping information
- Longer description (if necessary)
More to be added.
Schedule/milestones
General Litmus test cases in the Firefox 3 branch Accessibility group for current UI elements and to guard against regressions will be added on an ongoing basis. New UI elements for places, password manager, download manager, etc. are currently scheduled for beta 1. Appropriate Litmus test cases and object accessibility tests will be added ASAP.
Beta 1
General accessibility requirements have been met with this milestone. Many of the new UI features such as the Page Info dialog received a thorough accessibility review. There are some areas that remain keyboard-inaccessible, though:
- bug 400703 - Organize, Views, and Import and Export buttons not accessible via keyboard in Places Organizer
- bug 403956 - Provide a shortcut to directly open the Places folder.
- Larry UI not keyboard accessible.
In addition, one major bug for many screen reader users remained, covered in bug 391490 - JAWS reading all options for comboboxes in virtual mode, this is not usable.
Beta 2
These bugs were fixed for beta 2 in the Core area of Disability Access APIs: [1]. Of these, the following should be verified through manual testing:
These are verified OK for Linux, Windows XP and Vista.
Known issues (these should be folded into the release notes):
- Various issues with popups like the new AutoComplete popup on the location bar, the Add Bookmarks dialog (CTRL+D), the Download Manager's Info button. Some screen readers get confused by these panels announcing themselves as "menu". This issue is dealt with in bug bug 407359, but only for beta 3.
- Larry UI not keyboard accessible. This will also be dealt with in bug 407359.
- Site button has no keyboard shortcut, have to use Tools/Page Info dialog, Security and Privacy tab to get info.
Firefox 3 Beta 3
These are bugs fixed for Firefox 3 Beta 3: Buts fixed in Beta 3. The following were verified through manual testing:
- bug 407359 now works with all except one screen reader, also addresses bug 393398. Screen readers no longer announce "menu" when selecting an item in the AutoComplete popup, or when adding a bookmark with CTRL+D.
- This also fixes bug 400888. This should be described in the RelNotes for Beta 3 as follows: "The Site button can now be reached with the keyboard. To get to it, press CTRL+L or ALT+D for the Location bar, then press SHIFT+TAB to reach the button for the most important Site button information. After pressing it, a link becomes available that allows you to view more information in the Page Info dialog."
- bug 405679 - Missing content on WordPress.org search page. This also fixed bugs bug 408997, bug 401395 and bug 394493.
- Various AccessKey related bugs were also fixed, see bugs bug 409623, bug 413658, bug 413661, bug 398689, bug 410973, bug 408354, bug 410663, bug 411010, bug 408359, bug 402104.
- For DHTML authors, the ARIA namespace support has been completely removed bug 398910, so any content should be updated accordingly with this change.
- For RelNotes, a lot of crashers have been fixed in Core. Also, a major memory leak was fixed on Linux.
References
- The IAccessible2 Standard
- ATK: The Linux Accessibility Toolkit
- AT-SPI C Bindings Reference Manual
- Accerciser: An Interactive Python Accessibility Explorer for the GNOME Desktop
--Tim K 13:46, 20 July 2007 (PDT)
