B2G/QA/Meetings/2015-10-28

From MozillaWiki
< B2G‎ | QA‎ | Meetings
Jump to: navigation, search

NOTE: Please be aware it's a public etherpad and be careful about confidential information

// For 2015/10/28

Overview

   Meeting Name: B2G QA Roundtable

   Meeting Time (Taipei-EU): Wednesdays @ 3:30pm - 5:45pm CST

   Meeting Time (Taipei-MV/SF): Wednesdays @ 5pm - 5:45pm PDT

   Vidyo Room: QA-B2G Vidyo Room

   Video Link: https://v.mozilla.com/flex.html?roomdirect.html&key=A97nLDtY921vuLGRB1jv5S6OKs

   Dial in: 1-800-707-2533, #92, #9245

   Meeting Notes: https://wiki.mozilla.org/B2G/QA/Meetings

   Testrun Schedule: http://bit.ly/1jQzW79

   Roadmap for releases: https://wiki.mozilla.org/B2G/Roadmap

   B2G dashboard: http://charts.mozilla.org/

   QA Roundtable Template:  https://etherpad.mozilla.org/b2g-qa-roundtable-template  (Clone this)

   Performance result: http://raptor.mozilla.org


Announcements:

   List your ideas for Topics for Moz Orlando work week : qa@mozlando

   same idea as what we did for work week for Whistler

   Al has submit the meeting to google submit on Wednesday

   Next milestone: Oct. 30th - Finalize the meeting topics for FxOS teams


PTO plan

   Gerry+Mike+Shako+Walter+Shing - Oct. 30~Nov. 6 (Euro Star)

   marcia traveling 10-21 and 10-27


2.2r

   Contact Window: Shing Lyu

   Status: 

   Talked with 2 potential QA outsource candidate, waiting for their formal quote and proposal.

   LG provided us the RedTai source code, build ok but flashing falied.

   Under evaluation: pvt to taskcluster build

2.5 Feature

  • Telemetry: Iris/Gerry
    • Test plan draft: https://goo.gl/6ANJuO
    • Test cases has ready
    • Executing test cases in progress
    • Bug list: https://goo.gl/GXRnIH, no blocker
    • User stories: code have landed
      • bug 1181295 - FTE UX changes for privacy, landed
      • bug 1208205 - Settings changes for privacy, landed
      • bug 1177099 - compression, landed
    • Follow up bug:
      • bug 1215654 - Metrics may not be recorded when level set to enhanced, landed
  • Private Browsing: Alison
  • TV - Bookmark Sync: Cynthia
    • Testing in progress


  • Late Customization: Teri
    • Bug 1180707 - (2.5_LateCustomisation) late customisation
    • Testing in progress


   *** Bug list: http://mzl.la/1GoYtvy

   *** Risky to land this time
   *** Bug 1217682 landed but addons.allizom.org isn't up yet to do testing against:
** Silent Update
   *** Not in v2.5
  • Performance: Mike
    • App/Device startup time
   *** One regression from 20th Oct. https://bugzilla.mozilla.org/show_bug.cgi?id=1216403
    • Media performance still not landed yet
   *** https://bugzilla.mozilla.org/show_bug.cgi?id=1159343
   *** ETA: end of Wed. or it's too risky to test
    • Display/Graphic performance
   *** Regression test done and doesn't have new issue.

   *** Risky to land this time
   *** Process switch: https://bugzilla.mozilla.org/show_bug.cgi?id=1214572
   *** Data jar isolation: https://bugzilla.mozilla.org/show_bug.cgi?id=1178526
    • Testing in progress
   *** Test nsec packages: http://people.mozilla.org/~mlien/index.html
   *** Bug list: http://mzl.la/1GvtYEQ


_* NGA: Johan

    • Contacts work done, for 2.5 (split between front-end and back end, view split has been removed). SMS team still fixing 1 functional blocker and fixing perf. Music has no remaning blocker and has good confidence to overpass the performance that we had in 2.2.

Calendar work has not landed for 2.5. No plan

https://bugzilla.mozilla.org/show_bug.cgi?id=1215635 Notification bugs, seems there was a problem with the notification server but is fixed. Notifications are arriving but want to check today with more users that were not receiving the notifications


  • RTL: William
   • Development progress: (Implementation order: http://bit.ly/1MsORSC)
       - Completed:

   Dialer, FTU, System, Homescreen, Browser, Emergency Call, Settings, Contacts, Search, Keyboard, Bookmark

        - On-going:

   Email, Callscreen, Calendar

   • Test progress:
        - Completed:

   System (Second test run), FTU, Homescreen,  and Dialer RTL test runs.

        - Ongoing:

   SMS, Search, Keyboard, and Settings

        - Todo:

   E-mail and Contact (Patches are waiting for reviewing)

        - Test results: https://goo.gl/EgHFDm  
        - Bug list: https://goo.gl/KGvwvo  
   • What’s next?
        - We will start to triage RTL bug.
        - QA will help on E-mail and Contact test runs.
  • updates: FOTA : nhirata
    • Aries FOTA
      • Testing blocked by bug 1215667; IMEI whitelist work blocked.
    • Flame FOTA : gecko/gaia only
      • FOTA fixed using pvtbuild/buildbot; need to move to taskcluster (see agenda items)
      • T2M build being sent to T2M is blocked by waiting for information in regards to https://bugzil.la/1218600 / bug 1214222
  • Vaani : nhirata -
    • dropped for 2.5; work still being done, QA still needed. Will be an ongoing tweak of fuzzy logic
    • need vaani automatable: bug 1219450
    • please use the MozSpeech app in dogfood-latest build
    • automation talks with Johan and Andre. Waiting for Andre to make some changes to Vaani/API for reading in a recorded file.
  • Others
    • View Source: No-Jun
      • All major features are verified, there is only one bug that's being checked right now, but it's not a blocker (Bug 1204011)
      • Test suite is in moztrap

=== 2.5 Spark/Aries-KK Smoketests === (Read Only)

  • Tester: QAnalysts (KevinT)
    • Health Status: Yellow. 1 blocker

New:

- https://bugzil.la/1219161 [FTU] statusbar icons do not show up in the FTU 

* Fails [6119]
**Currently resolved as fixed. Should be verified in the next build.

Existing:

None

Verified as fixed: https://bugzil.la/1218439 Apps display improperly in Landscape orientation, homescreen wallpaper visible on right side of screen. 

* Failed [6073][6116][1333]
Closed as worksforme:

- https://bugzil.la/1219335 User is unable to receive OTA updates 

* Fails [2313]
    • This issue resolved itself somehow so it was closed.

Severe Issues: - https://bugzil.la/1217928 [New Homescreen] Homescreen App order is not preserved during OTA when moving to the New Home Screen - https://bugzil.la/1217220 Received videos through MMS do not play and a serious error occurs when trying to watch them in video app - https://bugzil.la/1208808 Discrepancy between toLocaleFormat and toLocaleString when timezone is unresolved - https://bugzil.la/1211266 Occasional missing icons after flashing homescreen - https://bugzil.la/1213233 [ Homescreen] All icons on homescreen can't be shown, after user selected "New Home Screen" or "Default Home Screen"

=== 2.5 Aries Dogfood === (Read Only)

  • Health Status: Aries: Green. No blockers

No blockers

Severe Issues: - https://bugzil.la/1217928 [New Homescreen] Homescreen App order is not preserved during OTA when moving to the New Home Screen - https://bugzil.la/1217220 Received videos through MMS do not play and a serious error occurs when trying to watch them in video app - https://bugzil.la/1208808 Discrepancy between toLocaleFormat and toLocaleString when timezone is unresolved - https://bugzil.la/1211266 Occasional missing icons after flashing homescreen - https://bugzil.la/1213233 [Homescreen] All icons on homescreen can't be shown, after user selected "New Home Screen" or "Default Home Screen"

=== 2.5 Flame FOTA === (Read Only)

  • Health Status: Flame Base: Green. No blockers

No Blockers

Issues Encountered:(Read Only) - https://bugzil.la/1218600 Dialer will crash when going to call log after FOTA/Sideload with a large call log 

* Fails [14042]

2.5 Exploratory

=== 2.5 fullrun (Marigold) === (William, update: 10/28) - 4804passed, 55 failed, 12 blocked,467 invalid / 6566 cases total. Coverage is at 96.10% - https://goo.gl/bs8lM0 ( 43 new bugs in 2.5-3 test run ) - https://goo.gl/3GnRe4 (12 permafail ) - https://goo.gl/PJzK17 ( 16 regressions )

=== 2.5 smoketest (Marigold) === (William, update: 10/28)

  • Aries
   1. https://bugzil.la/1218439, Bug 1218439 - [Aries] Apps display improperly in Landscape orientation, homescreen wallpaper visible on right side of screen
  • Flame
   1. None
  • Severe Issue: (Aries and Flame)

- https://bugzil.la/1217928 [New Homescreen] Homescreen App order is not preserved during OTA when moving to the New Home Screen - https://bugzil.la/1217220 Received videos through MMS do not play and a serious error occurs when trying to watch them in video app - https://bugzil.la/1208808 Discrepancybetween toLocaleFormat and toLocaleString when timezone is unresolved - https://bugzil.la/1211266 Occasional missing icons after flashing homescreen - https://bugzil.la/1213233 [Homescreen] All icons on homescreen can't be shown, after user selected "New Home Screen" or "Default Home Screen"


2.5 MTBF

flame-kk 512MB avg. running hours: 30.25 Blocker: -[wip] https://bugzil.la/1217289 Leak browser iframe through BrowserContextMenu and PinPageSystemDialog 

-  https://bugzil.la/1206634  [MTBF] After 20 hours executing, test cases failed consequently
aries-kk 2GB running hours: 48.09

mtbf bugs in Q4 https://bugzilla.mozilla.org/show_bug.cgi?id=1182954

    • RA exit criteria: 100 hrs
    • CC exit criteria: 200 hrs

Previous action items

(Update: Oct. 7th)

  • -DONE [marcia] Action item to follow up with Michael Treese in Orlando
  • [team] Finalize sessions/Items for Agenda for Orlando

https://docs.google.com/document/d/1B3zIDd2s3-kqMRqzefNxDNQ8aQGs3jTo9u0XskobNc4/edit

  • [Al/Pallavi] To follow up the QA ww between Moz ww
  • [In progress] marcia: follow up with pallavi regarding community budget.


Agenda

[nhirata] update 10/28:


[nhirata]: update 10/20

  • 2.5 Branching decision has been made; we will have to support 2 different branches
    • we need to figure out how to divy the work
  • pvtbuild server may be going away as early as early november
    • we need to figure out all bugs that block this. Is there a tag we should use? nhirata to organize all the bugs together for taskcluster team : QA whiteboard : [taskcluster-blocker]
    • How to perform regression windows?
      • mozregression tools
      • [jlorenzo - update after the meeting] One way we could do is:
   you get the HG (not git) commit from mozilla-central or inbound

   you go to https://tools.taskcluster.net/index/artifacts/#gecko.v1.mozilla-central.revision.linux/gecko.v1.mozilla-central.revision.linux 

   You do a ctrl+f on the gecko revision you

OR you can get a build from its build ID https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-central.nightly.2015.10.27/gecko.v2.mozilla-central.nightly.2015.10.27 For b2g-inbound: https://tools.taskcluster.net/index/artifacts/#gecko.v2.b2g-inbound.pushdate.2015.10.28/gecko.v2.b2g-inbound.pushdate.2015.10.28

        • [JMercado] - QAnalysts does something similar using the gecko hash from the sources.xml currently found in the pvt builds. Even that is quite slow and getting the commits from hg was slower. It would also be nice to somehow find builds by gaia commit if date/time indexing is a problem.
  • nexus builds can be publically available given the right base build. Talking about details with Elvin about builds

2.5 nexus-4 kk and nexus-5-l

[atsai]: update 10/21

  • move google doc to etherpad.
    • Please be reminded for partner confidential topics, please use qa-b2g-internal mail thread to sync up/discuss.


(update October 14))

https://gaia-l10n.allizom.org/integration/gaia-central/rev/e96ce12ed450 (Not shown in the diff, but you don't have anything to do about it)

  • Foxfooding [marcia]
    • Are people in the team actively foxfooding?
    • If so, can you help answer questions that are sent to the mailing list?


- Firefox Sync strings will likely be covered by automation, but needs to be confirmed as it seems not everything is covered

  • [Jamie_] Test day info, also a few questions.


  • Scope of this meeting: Is it supposed to be public or private?
    • If private: Remove this event from the public QA calendar, and remove the notes the public wiki
    • If public: Use a public etherpad for the meeting notes

Community Updates [marcia-read only]

[Jamie_]

  • will be starting on revamp of firefox OS QA pages soon, have to talk to some people about it.


This Week - Decisions & Action Items

  • [marcia] remove Pin the Web from Test Day



Testing information

Retrieved from "https://wiki.mozilla.org/index.php?title=B2G/QA/Meetings/2015-10-28&oldid=1103444"