Mobile/Testing/10 21 15

Previous Action Items

• snorp - investigate when to disable the webappstartup test. (bc will send email)

Status reports

Rel Eng

• bug 1201236 run tests on Try to generate seta data for Android chunking in bug 1183877 allowed me to enable all the remaining Android 4.3 debug tests in bug 1183877 Increase total-chunks for Android 4.3 Debug crashtests, js-reftests, and reftests.

A Team

• android-2-3-armv7-api9 Total failure rate: [1.46%]
  • last week 0.86%
  • total failure excluding retries 1.46%
• android-4-3-armv7-api11 Total failure rate: [2.53%]
  • last week 2.02%
  • total failure excluding retries 1.89%

Data fetched for: 2015-10-13 10:01 — 2015-10-19 19:56

• bug 1213938 accounts for about 50% of the 4.3 failures; now fixed - thanks :roc!

Autophone

Talos Test conversion

• ran tests over the weekend on jmaher's local nexus-s
  • rck3 is not so stable
  • tp4m/rck3 is very noisy
  • patch bug 1216636 to increase iterations, ignore first data point, and use the geometric mean of the remaining 5 up for review
  • patch bug 1216636 to allow for rck3 to miss a data point and retry (seemingly the cause of many orange rck3 runs)
  • what this looks like on treeherder
  • will plan to run again over the weekend and look for stability in tests and numbers.
  • next week plan to load this up on the nexus-7 and look at results there

Perfherder

• Going to look into putting apk size into perfherder this week or next (https://bugzilla.mozilla.org/show_bug.cgi?id=1149164)

Action Items

• snorp to file a bug for geckoview-on-ios testing
  • bug 1217097