FirefoxOS/DeviceQA/2015-02-09Status

V2.2 Highlight & Risk

• Highlight
  • Will start smoke test(auto, manual) on Nexus 5 Lollipop once we have stable PVT builds, currently builds are not yet created for v2.2 and builds from master are without image files.
    • Bug 1111890 - [Nexus 5] Create Lollipop based pvtbuilds for 2.2

• Day Off Notice
  • 2015/02/18~2015/02/23 CNY Holidays in Taiwan

V2.2 Milestone

• 2015/02/23 V2.2 FL
• 2015/04/06 V2.2 FC

• QA Shared calendar
  • You can add it to your Google Calendar view by clicking on this link and then clicking on the small blue "+ Google Calendar" button at the bottom, right hand of the screen.
  • If you are signed in with your @mozilla.com account, you can also access it this way:
    • In Google Calendar, you should go to Other calendars > Browse Interesting Calendars > More > Resources for mozilla.com > Public (9) > Public - FxOS QA Schedule > Subscribe

V2.2 Testing Scope

• Focus on bugs in agreed list.
• B2G V2.2 Overview Page for more info.
• Current feature-b2g=2.2+ 53 bugs
• Current blocking-b2g=2.2+ 76 bugs

V2.2 Testing by Type

Smoke Testing

On Track

• 3 blockers.
  • Bug 1128618 - [Windows Management] The header area will overlap the notification bar icons when picking a Gallery or Video app attachment for MMS or Email
  • Bug 1121045 - [b2g] we need root access on the b2g builds ( 3.0 & 2.2) for flame-kk
  • Bug 1118711 - Email app fails to fully launch due to worker JS runtime problem (reported as "InternalError: too much recursion") breaking startup process
• Running on Flame-KK

• Existing Bugs Breaking the Smoketests:
  • [Disabled] test_setup_and_send_active_sync_email.py - Bug 1028192 - Problems with ActiveSync accounts
  • [Disabled] test_receive_active_sync_email.py - Bug 1028192 - Problems with ActiveSync accounts
  • [Disabled] test_cost_control_data_alert_mobile.py - Bug 1061698 - Need ability to work in the shadow root/shadow DOM
  • [Disabled] test_call_log_all_calls.py - Bug 1096385 - Investigate failure in test_call_log_all_calls.py, call log is displayed erroneously

• Daily Reports can be found here.
  • Manual - are available on Google Drive & fxosqa-report-watchlist.
  • Automation - are available on Github.IO & fxosqa-report-watchlist.

Functional Testing

Yellow

• QA focus - 33 bugs found.
• Privacy (Gerry)
  • Health: Red
  • Test case coverage:
  • Details: 5 blockers
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1093688
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1111992
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1120577
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1120733
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1121232
      •  :bajaj to help send an email back to dave huseby

• Cut, Copy, and Paste (Gerry, Edward, Teri, Eric)
  • Health: Green
  • Test case coverage: 80%
  • Details: WIP

• RTL (delphine/Josh)
  • Health: yellow
  • Test case coverage: 30% (9/26) (William and Parul help to review TC)
    • Need Firefox Account and Market Place
    • Loop Delphine with the mail from Catalin
  • Details:
    • WIP https://wiki.mozilla.org/RTL#Test_cases_status_of_RTL
  • RTL feature-b2g 2.2+:
    • http://mzl.la/1JTN3vC
    • Josh send mail to EPMs with RTL bugs no owner

• NFC (Alison)
  • Health: Green
  • Test case coverage: 100%
  • Details:
    • v2.2 WIP
    • 3 blockers (regression bugs)
      • https://bugzilla.mozilla.org/show_bug.cgi?id=1120849
      • https://bugzilla.mozilla.org/show_bug.cgi?id=1124556
      • https://bugzilla.mozilla.org/show_bug.cgi?id=1125026

• Search suggestion based on provider (Hermes)
  • Health: Green
  • Test case coverage: 90%
  • Details: WIP

• App-grouping (Hermes)
  • Health: Green
  • Test case coverage: 99%
  • Details: WIP

• Task Manager (Hermes)
  • Health: Green
  • Test case coverage: 100%
  • Details: WIP
    • 1 blocker after landscape mode :
      • https://bugzilla.mozilla.org/show_bug.cgi?id=1128376

• L-porting (Hermes)
  • Health: Green
  • Details:
    • pvtbuild: Chris AtLee is handling. https://bugzilla.mozilla.org/show_bug.cgi?id=1111890
    • 2 components not finished
      • NFC is enabled this week, will test it.
      • Camera recording: https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1120780
      • https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1128836
        • • System was installed to data, and got wiped; turns out that it's not a recovery issue and the bug is resolved. ( build config has to include B2G_SYSTEM_APPS=1 )
          • Brian also follows up with Shawn, Keven

• CAF bugs (Eric)
  • Health: Green
  • Test case coverage: 100%
  • Details: Ready to test on regression

• TCL bugs (Alison, Eric)
  • Health: Green
  • Test case coverage: 100%
  • Details:
    • one blocker for feature [bug 1091544]: https://bugzilla.mozilla.org/show_bug.cgi?id=1123567
    • Alison to follow-up on this..

• Bluetooth (Teri)
  • Health: Green
  • Test case coverage: 100%
  • Details: Prepare to start regression testing

• Metrics (Shing)
  • Health: Yellow
  • Test case coverage: 100%
  • Details: 1 WIP https://bugzilla.mozilla.org/show_bug.cgi?id=1109422

• 256M RAM memory optimization for WFVGA, 5MP. (Teri)
  • Health: Green
  • Test case coverage: n/a
  • Details: Backed off
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1082290
    • Gaia UI test with and without patch on nexus-5 L in loop

• Multicore performance (Edward)
  • Health:
  • Test case coverage:
  • Better performance on Multicore (App startup time for core apps is comparable to Android

Exploratory Testing

On Track

• 202 bugs found - includes every bug in bugzilla affecting v2.2, not limited to the bugs after branch day.
  • UNCONFIRMED, NEW, ASSIGNED, REOPENED - 109 bugs found.
  • UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 8 bugs found.
  • RESOLVED, VERIFIED, CLOSED - 107 bugs found.
  • RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 6 bugs found.

• Regression
  • UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 22 bugs found.
  • RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 161 bugs found.

Performance Testing

• Will get started asap

Stability Testing

• Will start MTBF on Nexus 5 Lollipop once we have stable PVT builds AND entering FL.
• Bug 1111890 - [Nexus 5] Create Lollipop based pvtbuilds for 2.2

V2.2 Test Case Creation

Test Case Creation WIP

On Track

• [2.2-feature-qa+] & in-moztrap? (6 bugs) --> WORK IN PROCESS
• feature-b2g=2.2+ & in-moztrap? (0 bugs) --> WORK DONE

Test Case Creation DONE

• [2.2-feature-qa+] & in-moztrap+ (15 bugs) --> WORK DONE
• feature-b2g=2.2+ & in-moztrap+ (56 bugs) --> WORK DONE

V2.2 Testing Verification Status

Overall Verification Summary

Need Verification

30 Total; 0 Open (0%); 30 Resolved (100%); 0 Verified (0%);

Verification Done

1 Total; 0 Open (0%); 0 Resolved (0%); 1 Verified (100%);