FirefoxOS/DeviceQA/2015-02-02Status

V2.2 Highlight & Risk

• Highlight
  • Will start smoke test(auto, manual) on Nexus 5 Lollipop once we have stable PVT builds.
    • Bug 1111890 - [Nexus 5] Create Lollipop based pvtbuilds for 2.2

• Day Off Notice
  • 2015/02/18~2015/02/23 CNY Holidays in Taiwan

V2.2 Next Milestone

• 2015/02/23 V2.2 FL
• 2015/04/06 V2.2 FC

You can add it to your Google Calendar view by clicking on this link https://www.google.com/calendar/embed?src=bW96aWxsYS5jb21fMmQzNTMxMzYzMTMzMzUzNTM5MmQzNjMxMzBAcmVzb3VyY2UuY2FsZW5kYXIuZ29vZ2xlLmNvbQ and then clicking on the small blue "+ Google Calendar" button at the bottom, right hand of the screen.

If you are signed in with your @mozilla.com account, you can also access it this way: In Google Calendar, you should go to Other calendars > Browse Interesting Calendars > More > Resources for mozilla.com > Public (9) > Public - FxOS QA Schedule > Subscribe

V2.2 Testing Scope

• Focus on bugs in agreed list.
• B2G V2.2 Overview Page for more info.
• Current feature-b2g=2.2+ 63 bugs
• Current blocking-b2g=2.2+ 80 bugs

V2.2 Testing by Type

Smoke Testing

On Track

• No new blocker.
• Running on Flame-KK

• Existing Bugs Breaking the Smoketests:
  • [Disabled] test_setup_and_send_active_sync_email.py - Bug 1028192 - Problems with ActiveSync accounts
  • [Disabled] test_receive_active_sync_email.py - Bug 1028192 - Problems with ActiveSync accounts
  • [Disabled] test_cost_control_data_alert_mobile.py - Bug 1061698 - Need ability to work in the shadow root/shadow DOM
  • [Disabled] test_call_log_all_calls.py - Bug 1096385 - Investigate failure in test_call_log_all_calls.py, call log is displayed erroneously

• Daily Reports can be found here.
  • Manual - are available on Google Drive & fxosqa-report-watchlist.
  • Automation - are available on Github.IO & fxosqa-report-watchlist.

Functional Testing

On Track

• QA focus - 33 bugs found.
• Privacy (Gerry)
  • Health: Red
  • Test case coverage:
  • Details: 5 blockers
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1093688
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1111992
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1118155
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1120733
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1121232
      • [AI] Bhavana to help with this
• Cut, Copy, and Paste (Gerry, Edward, Teri, Eric)
  • Health: Green
  • Test case coverage: 80%
  • Details: WIP
• RTL
  • Health: Amber
  • Test case coverage: 57% (15/26)
    • Details: WIP https://wiki.mozilla.org/RTL#Test_cases_status_of_RTL
    • RTL feature-b2g 2.2+: http://mzl.la/1JTN3vC
• NFC (Alison)
  • Health: Green
  • Test case coverage: 100%
  • Details:
    • 2.2 features WIP
    • 3 blockers (regression bugs):
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1120849
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1125019
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1125026
• Rocketbar refactor (Hermes)
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1099157 has been removed from v2.2? wait confirmation from Candice.
• Search suggestion based on provider (Hermes)
  • Health: Green
  • Test case coverage: 80%
  • Details: WIP
• App-grouping (Hermes)
  • Health: Green
  • Test case coverage: 90%
  • Details: WIP
• Task Manager (Hermes)
  • Health: Green
  • Test case coverage: 100%
  • Details: would be finished after disable landscape mode (https://bugzilla.mozilla.org/show_bug.cgi?id=1118390)
• L-porting (Hermes)
  • Health: Yellow
  • Details:
    • a new bug resulted in build failed (https://bugzilla.mozilla.org/show_bug.cgi?id=1126499)
    • 1 Camera recording blocker: (https://bugzilla.mozilla.org/show_bug.cgi?id=1120780)
    • Other components are almost done.
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1111890#c11
• CAF bugs (Eric)
  • Health: Green
  • Test case coverage: 83%
  • Details: working on test case creation Bug 1043558
• TCL bugs (Alison, Eric)
  • Health: Green
  • Test case coverage: 100%
  • Details:
    • Feature "bug 1091544" has one blocker: https://bugzilla.mozilla.org/show_bug.cgi?id=1123567
• Bluetooth (Eric, Teri)
  • Health: Green
  • Test case coverage: 100%
  • Details: Full flash needed for bluetooth to work, waiting for nexus-l
• Metrics (Shing)
  • Health: Green
  • Test case coverage: 100%
  • Details: 1 backed out. No new feature.
    • Bhavana to follow-up and Shing will get testcases ready in the meantime
• 256M RAM memory optimization for WFVGA, 5MP. (Teri)
  • Health: Green
  • Test case coverage: cancelled
  • Details: code backed out due to bug 1081871.
• Multicore performance (Edward)
  • Health:
  • Test case coverage:
  • Better performance on Multicore (App startup time for core apps is comparable to Android

Exploratory Testing

On Track

• 202 bugs found - includes every bug in bugzilla affecting v2.2, not limited to the bugs after branch day.
  • UNCONFIRMED, NEW, ASSIGNED, REOPENED - 114 bugs found.
  • UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 12 bugs found.
  • RESOLVED, VERIFIED, CLOSED - 88 bugs found.
  • RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 2 bugs found.

Performance Testing

• Will get started asap

Stability Testing

• Will start MTBF on Nexus 5 Lollipop once we have stable PVT builds AND entering FL.
• Bug 1111890 - [Nexus 5] Create Lollipop based pvtbuilds for 2.2

V2.2 Test Case Creation

Test Case Creation WIP

On Track

• [2.2-feature-qa+] & in-moztrap? (8 bugs) --> WORK IN PROCESS
• feature-b2g=2.2+ & in-moztrap? (2 bugs) --> WORK IN PROCESS

Test Case Creation DONE

• [2.2-feature-qa+] & in-moztrap+ (15 bugs) --> WORK DONE
• feature-b2g=2.2+ & in-moztrap+ (41 bugs) --> WORK DONE

V2.2 Testing Verification Status

Overall Verification Summary

• No bugs marked as "VERIFIED FIXED" Yet.
• More details.

As in 2015/01/15 V2.2, Verification Status