QA/Reading list

From MozillaWiki
< QA
Jump to: navigation, search

Desktop Reading List

Ownership

Developer contacts: Blair McBride, Justin Dolske, Drew Willcoxon

QA Lead: Andrei Vaida, :avaida on IRC

Status

Overview

[LANDED] Nightly 38: first landed, disabled by default

[LANDED] Aurora 38: landed, currently enabled by default

[LANDED] Beta 38: landed, disabled by default

[ON TRACK] Release 38: tbd

Timeline

  • UPDATE 2015-03-22: the pref is true by default on 39.0a1.
  • UPDATE 2015-03-26: the pref is true by default on 38.0a2.
  • UPDATE 2015-03-30: the pref is true by default on 38.0b1.

Risk analysis

Event Probability Impact Current situation Mitigation steps
Adding a website to Reading List fails to work. probable high There were isolated cases where the add action failed. Periodical regression testing covers this area.
Opening a website from the Reading List fails to work. improbable high There were no failures reported for this case. Periodical regression testing covers this area.
Removing a website from Reading List fails to work. probable medium There were isolated cases where the remove action failed. Periodical regression testing covers this area.
The Reading List sidebar toggle button fails to work. improbable high There were no failures reported for this case. Periodical regression testing covers this area.
Local reading lists are not saved properly and some/all added items are missing. improbable high There were no failures reported for this case. Periodical regression testing covers this area.
Reading List items are not synced up with the Firefox Account. probable medium There were isolated cases where the sync-up action failed. Periodical regression testing covers this area.
Reading List items are not synced down from the Firefox Account. probable high There were isolated cases where the sync-down action failed. Periodical regression testing covers this area.

Summary

Context

Reading List for Firefox Desktop is based on a cross-product set of features, scheduled for release with Firefox 38.

The features planned for Reading List v1 are:

  • a synced reading list for a user signed into a FxA
  • a local only reading list for a user not signed into a FxA
  • a Reader Mode that integrates with the Reading List for reading and saving

What will not be available in Reading List v1:

  • offline reading support, currently available only on mobile
  • web access independent of Firefox (cloud based access)

Enabling the feature

Reading List is available starting with Firefox 38 and it can be enabled by setting the following pref to true:

browser.readinglist.enabled

References

  • The UX planned for Reading List is available here.

Test cases

Overview

Targeted scenarios

Testing focused on basic functionality

  • Enable Reading List
  • Disable reading List
  • Check the pre-populated Reading List using a new profile
  • Add a web page to local Reading List using the Location Bar
  • Enable the Reading List sidebar
  • Remove a web page from local Reading List using the sidebar
  • Enable the Reading List sidebar from the Bookmarks Menu entry
  • Sync down the Reading List from a Firefox Account
  • Sync up local Reading List to a Firefox Account

Testing focused on interoperability

  • Add web page to Reading List from Reader View's sidebar (non-pb & pb)
  • Add web page to Reading List from Location Bar and restart Firefox (non-pb & pb)
  • Add web page to Reading List from Library
    • note: since the websites from Reading List are in some way similar to bookmarks, it might be possible to add a new website to the Reading List from the Library; same goes for removing a website from the list
  • Open a website from History and add it to Reading List (non-pb & pb)
  • Restore a closed tab from History and add it to Reading List (non-pb & pb)
  • Open a website, bookmark it and add it to Reading List (non-pb & pb)
  • Open a bookmarked website and add it to Reading List (non-pb & pb)
  • Bookmark a website from the Reading List (non-pb & pb)
  • Import bookmarks and add one to Reading List
  • Add a website to Reading List from context menu (non-pb & pb)
  • Merge local Reading List with a second Reading List associated to a Firefox Account
  • Merge two different Reading Lists associated to two different Firefox Accounts
  • Disable Reading List syncing from about:preferences#sync
  • Initiate manual sync after adding a website to Reading List
    • note: as in the case of Bookmarks, adding a website to Reading List might be designed to initiate up-syncing automatically
  • Remove pre-populated Reading List websites (non-pb & pb)
  • Open website from Reading List and display it in Reader View (non-pb & pb)
  • Add several websites to Reading List subsequently (at least 15)
  • Open several websites from the Reading List sidebar subsequently, in separate tabs (at least 15)
  • Pin a tab where Reading List sidebar is enabled

Testing based on recent bugs and regressions

  • Enable the Reading List sidebar on Firefox Developer Edition
  • Add web pages with lengthy titles to Reading List
  • Check how the Reading List and Reader View buttons are displayed in the Location Bar
  • Check how the Reading List button from the Location Bar changes according to its status
  • Attempt to add an in-content page to Reading List
  • Add a web page to Reading List from Reader View's controls
  • Check the Reading List button styling on Windows OS
  • Check the "Add page to Reading List" button styling on Firefox Developer Edition
  • Check the "Add page to Reading List" button styling on high-contrast themes
  • Add a web page to Reading List from a private window

Testing Reading List with e10s

All regular test cases already available in test suites will be executed with e10s enabled.

Bug work

Tracking bug

meta: Bug 1123529 - (desktop-readinglist) [meta] Reading List on desktop

Bug triage +/- for verification

[QE-VERIFY+] Bug 1137231 - Double Separator In Bookmarks Menu

↳ 2015-02-26: most likely caused by Bug 112440

[QE-VERIFY+] Bug 1124647 - [ReadingList] Implement reading list controls in the reader mode control bar

↳ 2015-03-06: qe-verify+ granted by Florin Mezei

[QE-VERIFY+] Bug 1142217 - Make the Add/Remove to Reading List toggle button of the reader view actually work
[QE-VERIFY+] Bug 1134497 - Implement infopanels for in-product FTU Reading List/View tour
[QE-VERIFY+] Bug 1134507 - Implement infopanels to promote Reader View / Reading List when first seen
[QE-VERIFY+] Bug 1143388 - Clicking reading list item from bookmarks menu should open it in reader view
[QE-VERIFY+] Bug 1141353 - Fix ReadingList bookmarks nativemenu on OSX
[QE-VERIFY+] Bug 1144380 - selected/active state on reading list sidebar persists after switching tabs
[QE-VERIFY+] Bug 1144377 - hover state on reading list sidebar items remains after the mouse leaves the list
[QE-VERIFY+] Bug 1144774 - Add to reading list button is blurry
[QE-VERIFY+] Bug 1144781 - Add to reading list button sometimes doesn't hide when mouseing out from the urlbar
[QE-VERIFY+] Bug 1144823 - reading list sidebar should be sorted by most recently added
[QE-VERIFY+] Bug 1144382 - title text in reading list sidebar item wraps poorly, leaving partial text visible
[QE-VERIFY+] Bug 1145372 - The add to reading list url bar button is larger than the other url bar buttons
[QE-VERIFY+] Bug 1145409 - Show the site favicon for each reading list item
[QE-VERIFY+] Bug 1146946 - TypeError: this.selectedItem is null from sidebar.js when using the up/down key when the list is empty.
[QE-VERIFY+] Bug 1147444 - Improve the transition when deleting an item from the Reading List
[QE-VERIFY+] Bug 1147440 - Improve the transition of the Add to Reading List URL Button
[QE-VERIFY+] Bug 1147479 - Improve the transition when adding an item from the Reading List
[QE-VERIFY+] Bug 1148364 - On changing an article the "Remove from Reading List"-Button flickers
[QE-VERIFY+] Bug 1148462 - When "Reading List" is disabled (browser.readinglist.enabled = false) CTRL+ALT+R should not open its sidebar.
[QE-VERIFY+] Bug 1148562 - If I right-click the reading list "+" button (in URL bar), the page gets added to my reading list
[QE-VERIFY+] Bug 1148201 - Fix desktop reading list client's use of Last-Modified, If-Unmodified-Since, If-Modified-Since headers
[QE-VERIFY+] Bug 1148240 - Reading list item is difficult to delete if the vertical scroll bar is visible
[QE-VERIFY+] Bug 1149869 - Repeatedly adding and removing an item from reader mode causes duplicate items to appear in the sidebar
[QE-VERIFY+] Bug 1148762 - Reading List not full but scrollable for 1px, causes scroll bar to flash when opening reading list sidebar
[QE-VERIFY+] Bug 1149823 - Keep reading list sidebar sorted when new items are added via sync
[QE-VERIFY+] Bug 1149023 - unable to remove reading list item
[QE-VERIFY+] Bug 1147409 - right click on sites in readlist cannot remove it
[QE-VERIFY-] Bug 1153121 - Write local items to the sync log if any logs have "Trace" level
[QE-VERIFY-] Bug 1153371 - Create shared place for readability content CSS styles
[QE-VERIFY+] Bug 1151823 - Reading list item is missing hover effect in "Sidebars" dropdown panel
[QE-VERIFY+] Bug 1152412 - Reading list items that are images get stuck in "loading" state
[QE-VERIFY-] Bug 1148701 - React to Backoff and Retry-After headers from Reading List storage servers

Bug fix verification

[VERI] Bug 1124400 - [ReadingList] Add section to bookmarks popup for reading list items and actions

↳ 2015-02-25: verified fixed on 39.0a1, across platforms; follow ups to be made according to Comment 12
↳ 2015-03-02: verified fixed on 38.0a1, across platforms

[NEW] Bug 1133611 - [ReadingList] Add promo blurb to ReadingList bookmarks menu
[NEW] Bug 1133610 - [ReadingList] Add promo blurb to ReadingList sidebar
[VERI] Bug 1133485 - Hook up "Add to ReadingList" button in desktop ReaderMode

↳ 2015-03-23: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[NEW] Bug 1131461 - [ReadingList] Show status text in the URLBar when interacting with the ReadingList button
[NEW] Bug 1131460 - [ReadingList] Add-to-readinglist button in the URLbar should have an animation similar to the bookmarking animation
[VERI] Bug 1131457 - [ReadingList] Add a button to the URLBar that allows adding the current page to the Reading List

↳ 2015-03-18: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1123517 - [ReadingList] Implement basic sidebar that lists unread ReadingList items

↳ 2015-02-25: verified fixed on 39.0a1 and 38.0a2, across platforms; potential issues reported

[VERI] Bug 1133662 - [ReadingList] Show something useful in the sidebar when the Reading List is empty

↳ 2015-03-20: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[NEW] Bug 1124153 - [ReadingList] Make the list of items in the Reading List sidebar scale well
[VERI] Bug 1123525 - [ReadingList] Allow deleting items via the Reading List sidebar

↳ 2015-03-20: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[NEW] Bug 1123519 - [ReadingList] Allow adding items to the Reading List via the sidebar
[VERI] Bug 1123518 - [ReadingList] Add thumbnails/favicons to items in the Reading List sidebar

↳ 2015-03-23: verified fixed on 39.0a1, across platforms; Bug 1146348 was filed as a follow-up
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1137231 - Double Separator In Bookmarks Menu when ReadingList is disabled

↳ 2015-03-02: verified fixed on 39.0a1 and 38.0a2, across platforms

[VERI] Bug 1124647 - [ReadingList] Implement reading list controls in the reader mode control bar

↳ 2015-03-09: verified fixed on 39.0a1 and 38.0a2, across platforms

[VERI] Bug 1133489 - Hook up "Open ReadingList" button in desktop ReaderMode

↳ 2015-03-09: verified fixed on 39.0a1 and 38.0a2, across platforms

[VERI] Bug 1142217 - Make the Add/Remove to Reading List toggle button of the reader view actually work

↳ 2015-03-16: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1144382 - title text in reading list sidebar item wraps poorly, leaving partial text visible

↳ 2015-03-23: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1144380 - active state on reading list sidebar persists after switching tabs

↳ 2015-03-23: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1144823 - reading list sidebar should be sorted by most recently added

↳ 2015-03-23: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1145372 - The add to reading list url bar button is larger than the other url bar buttons

↳ 2015-03-23: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1144680 - The Reading List button from the Location Bar is available for in-content pages

↳ 2015-03-23: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1144675 - The Reading List button from the Location Bar should have a distinct icon for pages currently in the list

↳ 2015-03-23: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1144377 - hover state on reading list sidebar items remains after the mouse leaves the list

↳ 2015-03-23: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1144781 - Add to reading list button sometimes doesn't hide when mouseing out from the urlbar

↳ 2015-03-23: verified fixed on 39.0a1, across platforms, via Bug 1144675

[VERI] Bug 1145911 - Enable reading list by default

↳ 2015-03-24: verified fixed on 39.0a1, across platforms
↳ 2015-03-30: verified fixed on 38.0a2, across platforms

[VERI] Bug 1143388 - Loading an item from the readinglist sidebar shouldn't make the user leave the reader mode if it was active

↳ 2015-03-30: verified fixed on 39.0a1 and 38.0a2, across platforms

[VERI] Bug 1146946 - TypeError: this.selectedItem is null from sidebar.js when using the up/down key when the list is empty.

↳ 2015-03-30: verified fixed on 39.0a1 and 38.0a2, across platforms

[VERI] Bug 1137556 - Make the ReadingList sidebar look like other sidebars

↳ 2015-03-30: verified fixed on 39.0a1 and 38.0a2, across platforms

[VERI] Bug 1146486 - The generic reading list thumbnail graphic is stretched too big and on white background

↳ 2015-03-30: verified fixed on 39.0a1 and 38.0a2, across platforms

[VERI] Bug 1146358 - The attempt of adding a page to Reading List from Reader View fails and causes an error

↳ 2015-03-30: verified fixed on 39.0a1 and 38.0a2, across platforms

[VERI] Bug 1147440 - Improve the transition of the Add to Reading List URL Button

↳ 2015-04-02: verified fixed on 40.0a1, across platforms
↳ 2015-04-07: verified fixed on 38.0b2-build1 (20150406174117), across platforms

[VERI] Bug 1147444 - Improve the transition when deleting an item from the Reading List

↳ 2015-04-02: verified fixed on 40.0a1, across platforms
↳ 2015-04-07: verified fixed on 38.0b2-build1 (20150406174117), across platforms

[VERI] Bug 1147479 - Improve the transition when adding an item from the Reading List

↳ 2015-04-02: verified fixed on 40.0a1, across platforms
↳ 2015-04-07: verified fixed on 38.0b2-build1 (20150406174117), across platforms

[VERI] Bug 1147889 - need transition when switching between light-dark-sepia

↳ 2015-04-02: verified fixed on 40.0a1 and 39.0a2, across platforms
↳ 2015-04-07: verified fixed on 38.0b2-build1 (20150406174117), across platforms

[VERI] Bug 1134446 - Automatically open the ReadingList sidebar the first time ReaderMode is used

↳ 2015-04-02: verified fixed on 40.0a1 and 39.0a2, across platforms

[VERI] Bug 1149869 - Repeatedly adding and removing an item from reader mode causes duplicate items to appear in the sidebar

↳ 2015-04-02: verified fixed on 40.0a1, across platforms
↳ 2015-04-10: verified fixed on 38.0b3-build1 (20150409144858), across platforms

[VERI] Bug 1137089 - The sidebar header is white on the developer edition theme

↳ 2015-04-02: verified fixed on 40.0a1 and 39.0a2, across platforms

[VERI] Bug 1149023 - unable to remove reading list item

↳ 2015-04-02: verified fixed on 40.0a1 and 39.0a2, across platforms
↳ 2015-04-07: verified fixed on 38.0b2-build1 (20150406174117), across platforms

[VERI] Bug 1148462 - When "Reading List" is disabled (browser.readinglist.enabled = false) CTRL+ALT+R should not open its sidebar.

↳ 2015-04-06: verified fixed on 40.0a1, across platforms
↳ 2015-04-07: verified fixed on 38.0b2-build1 (20150406174117), across platforms

[VERI] Bug 1144774 - Add to reading list button is blurry

↳ 2015-04-10: verified fixed on 38.0b3-build1 (20150409144858), across platforms

[VERI] Bug 1148201 - Fix desktop reading list client's use of Last-Modified, If-Unmodified-Since, If-Modified-Since headers

↳ 2015-04-14: verified fixed on the 38 and 39 branches, across platforms

[VERI] Bug 1148273 - Readinglist scheduler doesn't handle FxA error states correctly.

↳ 2015-04-14: verified fixed on the 38 and 39 branches, across platforms

[VERI] Bug 1149136 - Incorrect hover styling displayed for the article remove button available in the Reading List sidebar

↳ 2015-04-16: verified fixed on 38.0b4-build1, across platforms

[VERI] Bug 1149302 - Error when adding an item to reading list

↳ 2015-04-16: verified fixed on 38.0b4-build1, across platforms

[VERI] Bug 1152412 - Reading list items that are images get stuck in "loading" state

↳ 2015-04-16: verified fixed on 40.0a1, across platforms

[VERI] Bug 1148762 - Reading List not full but scrollable for 1px, causes scroll bar to flash when opening reading list sidebar

↳ 2015-04-29: verified fixed on 40.0a1, across platforms
Logged bugs

Bug 1137089 - The Reading List sidebar header is white on the developer edition theme

↳ 2015-02-26: found while verifying the fix pushed for Bug 1123517
↳ 2015-02-27: this is actually an issue affecting all sidebars on the Developer Edition theme, so it's not strictly related to the Reading List feature

Bug 1143689 - Longer item titles from the Reading List are displayed as cutoff in the sidebar

↳ 2015-03-16: found while verifying the fix pushed for Bug 1142217

Bug 1144666 - There's no separator displayed in the Location Bar between the Reading List and Reader View buttons

↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457

Bug 1144668 - Clicking the Add to Reading List button from the Location Bar does not trigger the Show Bookmarks icon animation

↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
↳ 2015-03-19: marked as DUPE of Bug 1131460 by Tim Nguyen

Bug 1144675 - The Reading List button from the Location Bar should have a distinct icon for pages currently in the list

↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457

Bug 1144680 - The Reading List button from the Location Bar is available for in-content pages

↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457

Bug 1144684 - The status of Reading List's button from the Location Bar is not syncing with the one from Reader View's controls

↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457

Bug 1144691 - Windows displays a blue glow hover effect for the Reading List button in the Location Bar

↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457

Bug 1144702 - The Reading List button from the Location Bar is not adapting to high contrast and Developer Edition themes

↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
↳ 2015-03-23: closed as DUPE of Bug 1134658, by Gavin Sharp

Bug 1144716 - Toggling the Reading List button from the Location Bar of private windows throws errors in the Browser Console

↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
↳ 2015-03-23: closed as WFM, as the error is no longer reproducible on 39.0a1

Bug 1145565 - The "Show Reading List Sidebar" button from the bookmarks menu should be visible at all times

↳ 2015-03-20: found while running Reading List test cases

Bug 1145609 - The Reading List item removal button should have the same tooltip as the one from the Location Bar

↳ 2015-03-20: found while verifying the fix pushed for Bug 1123525

Bug 1149136 - Incorrect hover styling displayed for the article remove button available in the Reading List sidebar

↳ 2015-03-30: found while signing off the feature for the merge to beta

Bug 1149144 - Crash-report page thumbnail displays a never-ending loading circle in Reading List sidebar

↳ 2015-03-30: found while signing off the feature for the merge to beta

Bug 1149902 - The Reading List's sidebar is sometimes enabled automatically in newly opened private windows

↳ 2015-04-01: found while verifying build1 of 38.0b1-build1

Bug 1150369 - Intermittently, syncing Reading List items fails with an error thrown by Sync.jsm:431

↳ 2015-04-02: found while verifying build1 of 38.0b1-build1
↳ 2015-04-02: marked as DUPE of Bug 1149105 by Mark Hammond

Bug 1150388 - Some items from Reading List are doubled after the automated syncing with Firefox Account

↳ 2015-04-02: found while verifying build1 of 38.0b1-build1

Bug 1151845 - Thumbnail and name for a previous played youtube video are added in Reading list

↳ 2015-04-07: found during the verification of 38.0b2-build1, by Petruta Rasa

Bug 1151871 - Links with the same domain can not be added to Reading List

↳ 2015-04-07: found during the verification of 38.0b2-build1, by Petruta Rasa

Bug 1151823 - Reading list item is missing hover effect in "Sidebars" dropdown panel

↳ 2015-04-07: found during the verification of 38.0b2-build1, by Catalin Varga

Bug 1152192 - Repeatedly adding and removing an item from Reader View causes duplicates in Reading List's sidebar

↳ 2015-04-08: filed as a follow-up issue for Bug 1149869

Bug 1152307 - Reading list is not synced

↳ 2015-04-08: found by Mihaela Velimiroviciu while testing Reading List with e10s enabled on 40.0a1

Sign off

Criteria

  • All the test cases were executed.
  • All the blocker, critical, major bugs have been fixed.

Results

Merge to Aurora Sign-off

  • [DONE] Aurora 38 (2015-02-27).
    • There were no blockers found while testing this feature.
    • Manual testing covered Windows 7 (x64), Windows 8.1 (x86), Ubuntu 14.04 (x64) and Mac OS X 10.9.5.

Merge to Beta Sign-off

  • [DONE] Beta 38 (2015-03-30).
    • There were no blockers found while testing this feature. Detailed test results are available in this etherpad.
    • Manual testing covered Windows 7 (x64), Windows 8.1 (x64), Windows XP (x64), Ubuntu 14.04 (x86) and Mac OS X 10.9.5.

Merge to Release Sign-off

  • [ON TRACK] RC 38.0 (date).
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Reading_list&oldid=1071279"