X2N Migration Test Plan

• QA Contact: Naoki Hirata IRC: irc://irc.mozilla.org#qa nhirata
• Releng Contact : Aki IRC: irc://irc.mozilla.org#releng aki
• Builds: 10.0.4 ESR XUL Build, Single/Multi | 13.0b1 XUL, Single/Multi | 14a2 Aurora (Beta Candidate)

Purpose : To verify that upgrading to Native 14 from neither 10.0.4 ESR XUL build nor 13 beta XUL build causes any issues, including but not withstanding localization issues. For more information, please see bug 750841 and bug 750819

• Note: Rel Eng (Aki) will be helping out and pushing the test builds to Google Play for testing purposes

Current Plan : Concentration on Beta Scenario

Current Status :

• QA to test bug 750819
• QA to test bug 750841; RelEng has builds for bug 750841. Need to coordinate with Releng once devices are ready to go

Results

• Beta Results
• Release Results

Scenarios

Current scenarios

Google Play Update Testing

Before Beta (750819, 747534, and "test scenario 1"):
* RelEng: Verify that a multi-locale XF for tablets can be uploaded alongside many single locale FN builds 
* QA: Determine what build different localizations will receive depending on Firefox/system locale and available builds 
* QA: Verify that FN14 is offered to phone XF13 users, XF14 is offered to tablet XF13 users, and no unexpected issues occur
* QA: Test the update experience for an Android 2.0 XF13 user, now that FN14 no longer supports them

Note: We ran into some issue with posting multiple locales up on Google Play. We will only be testing multi locale XUL fennec to Single locale English Native Fennec for the first test run for phones and multi locale XUL fennec to multi locale XUL fennec for tablets.

Before Release (750841, and "test scenarios 2/3"): <split APK>
* QA: Verify that FN14 is offered to phone XF10 users, XF14 is offered to tablet XF10 users, and no unexpected issues occur
* QA: Determine what build different localizations will receive depending on Firefox/system locale and available builds

Migration Testing (Google Play is unnecessary)

Before Beta (750819, builds available):
* QA: Verify that XF13 to FN14 migrations (APK over APK) have no new perf/functional/crash regressions across multiple devices

Before Release (748096):
* QA: Verify that XF10 to FN14 migrations (APK over APK) have no new perf/functional/crash regressions across multiple devices

Scenario 1 - tests XF13->FN14 split-apk and single-locale at the same time, in preparation for beta
We need:
* A test build of a multi-locale XF13, with the same localization set as Beta (en-US only right now)
* Test builds of FN14 with a newer versioncode, single-locale APKs in en-US, ja, and fr
* Test builds of XF14 with a newer versioncode, single-locale APKs in en-US, ja, and fr

Test Plan
* Upload multi-locale XF13
* Install multi-locale XF13
* Set Firefox language and system language (en-US/ja/fr) on phone/tablet
* Upload XF14 en-US, ja, and fr for tablets
* Upload FN14 en-US, ja, and fr for phones
* Install 14 on tablet
* Install 14 on phone
* Report back what language you got in FN, and any issues you ran into 

Scenario 2 - tests single-locale XF10.0.4->FN14 update experience, in preparation for release
We need:
* A test build of a multi-locale XF10.0.4, with the same localization set as Release
* Test builds of FN14 with a newer versioncode, single-locale APKs in en-US and ja

Test Plan
* Upload multi-locale XF10.0.4
* Install XF10.0.4
* Set Firefox language and system language to (English  on phone/tablet
* Upload FN14 en-US and ja
* Install 14 on phone
* Report back what language you got in FN, and any issues you ran into 

Scenario 3 - tests single-locale XF10.0.4->FN14 update experience when your language is initially unavailable, in preparation for release
We need:
* A test build of a multi-locale XF10.0.4, with the same localization set as Release
* Test builds of FN14 with a newer versioncode, single-locale APKs in en-US
* Test build of FN14 with an even newer versioncode, single-locale APK in en-US and fr

Test Plan
* Upload multi-locale XF10.0.4
* Install XF10.0.4
* Set Firefox language and system language to French on phone/tablet
* Upload FN14 en-US
* Install 14 on phone
* Upload second set of FN14 builds (fr and en-US)
* Install 14 on phone
* Report back what language you got in FN, and any issues you ran into