QA/Automation/Meetings/141117

From MozillaWiki
< QA‎ | Automation‎ | Meetings
Jump to: navigation, search

previous meeting | Meetings | next meeting »

Dial in

 # When:     Every Monday at 8:45am PDT/PST
 # Vidyo:    https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:    650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #           1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:      irc://irc.mozilla.org:6697/#automation
 # IRC logs: http://irclog.gr/#browse/irc.mozilla.org/automation

General

  • Attendees: Andreea, Henrik, Teodor, Cosmin, Mihaela, Marc, Barbara, Nils
  • Chair: Henrik
  • Notes: Andreea

Last Weeks Action Items

  • [NEW] [Henrik] follow up on bug 1083166
    • everything seems to work now, but we need to follow up here to get this investigated
  • [DONE] [Henrik] Ask regarding open squid proxy
    • Proxy host has been created, now up to networking team to setup ACLs
    • The VM has been setup, the requirements have been pushed to the team so we'll have this available soon
    • Once this has been landed, we can start to create tests for any network/proxy related testcases (we should check in-qa-testsuite)

Highlights

  • [Henrik] Firefox Automation report – week 37/38 2014: http://www.hskupin.info/2014/11/11/firefox-automation-report-week-37-38-2014/
    • Henrik will add 2 new reports this week on his blog
  • [Henrik] Finally we removed all traces for ESR24 jobs from tests and CI (needs deployment to production)
  • [Henrik] Mozmill-CI staging got updates: https://github.com/mozilla/mozmill-ci/compare/ebe8b456d7f6...1ee11e3b78cc (Ubuntu 14.04, esr24 removal)
    • We should remove ubuntu 12.04 once we confirm 14.04 works fine
  • [Mihaela] Memory consumption increase in endurance tests starting November the 11th - bug 1099145 Increased memory consumption starting Nov the 11th
    • all platforms affected, with a greater difference on Mac OS X 10.9.5
    • could not reproduce locally so far, today Mihaela tried on 10.9 and 10.8, with fewer iterations and entities
    • will continue with 10 iterations/entities as we run on our CI, and maybe continue on staging if locally nothing happens
    • Henrik: maybe the team from AWSY have detected this yet?
    • Andrei: crosscheck if it's related to the updates we've recently done: java, flash, vmware
    • Henrik: maybe correlate with staging/production results, if we see this on both
  • Henrik landed the upgrade of Jenkins, details below on project updates

Blockers

  • List of blockers (http://bit.ly/Tph0NY):
    • [07/10] - Bug 735184 - RFE: Create Pulse notifications for update channel activities (maybe in Q4)
    • [09/01] - Bug 1038646 - Broken package dependencies when installing OpenJDK 7 on Ubuntu 12.04 machines (maybe wontfix because we want to get rid of it)
      • Most likely we'll rely on openjdk, will be installed by default on linux
      • We don't use java on our mozmill tests, so we don't need oracle versions
    • [09/03] - Bug 997230 - Setup Mac Mini for DeployStudio needed by PuppetAgain (needs action RelOps)
    • [10/29] - Bug 917204 - Add open squid proxy to qa.scl3.mozilla.com for testing purposes (waiting setting up network ACLs)
    • [10/29] - Bug 1066493 - TPS failure: Error: signIn() failed with: "error.sync.reason.serverMaintenance" (depends on bug 1057892)
      • Waiting for the push on production for the database, hopefully by today/tomorrow will be landed and the failures will be fixed
    • [11/05] - Bug 1083166 - [SCL3] Pulse and maybe other connections are timing out in irregular intervals (maybe also causing test failures when loading web pages)
    • [11/07] - Bug 1094272 - [pulsetranslator] With SSL enabled publishing of normalized messages get stuck
    • [11/11] - Bug 1096367 - move aurora updates onto cdn temporarily
      • We might need to find a solution here, it happened only once, in one day, but it might happen again

Contributor Spotlight

  • Barbara Miller (:galgeek) will be our GNOME Outreach Program intern from December to March
    • she spent some time on mozmill tests, she got more interested on automation
    • the program is trying to help people in groups underrepresented around the world

Roundtable

  • TBD

Goals

Framework and Tools (Henrik)

Make Marionette ready for Mozmill tests

  • e10s support will most likely land for Firefox 37.0 by end of February 2015; we should have tests available when we go to Aurora at latest early Beta
  • Restart support landed for Marionette but we still miss the more important application triggered restart (see bug 1099127)
  • A couple more tests have been converted and examples created for feature checks, I will continue to do more myself this week
    • Will take a bit of time to be able to run mozmill tests on nightly, maybe try to do them for aurora and beta
    • Lots of work and enhancements for marionette for the ateam
    • David Burns was working on restart support, has been landed last week, we restart from inside the application, but stills represents only about 5% we actual need in our tests - blocking most of our important tests - like update tests

Locale Rotation for Aurora builds in Mozmill-CI

  • No updates

Turn on PuppetAgain for Mozmill CI and TPS CI

  • No updates

Improvements to Testday bot [stretched]

  • No updates

Test Automation (Softvision)

Creation of automated tests for important regressions

  • New tests in work:
    • bug 1088561 - Add automated test for plugins in different states | Almost done, only nits in last review.
    • bug 941477 - Create a remote Mozmill test for the 'Get more Search Engines' feature | Waits for review
    • bug 1096209 - Add automated tests for Firefox scrolling & zooming | Waits for review
    • bug 1088638 - Add new automated test to install, disable, enable and remove services la new tests | 50% done

Project Updates

Mozmill (Henrik)

  • A couple of patches landed for Mozmill CI on staging (will be pushed to production this week)
    • There's no count of test failures in the emails from the CI list, this issue was disabled for a while
    • Enabling Ubuntu 14.04 nodes, and disabling 12.04 (which will be removed soon)
    • Removal of 24.0ESR related jobs
    • Upgrade to Jenkins 1.580.1: https://github.com/mozilla/mozmill-ci/issues/523 including fixes for:
      • Huge HTTP session count over time
      • Re-enabling Junit reports without wait times when collecting results
      • Worked on the jenkins upgrade, huge patch, we need to enable it on staging
      • Monitor plugin also had an upgrade, was tested locally and we will do more testing on staging
    • Fixed a problem with XShell, which was not able to kill jobs anymore
    • we weren't able to kill the process anymore, after the abort on CI happened
  • Fixed Flashplugin-installer issue for download notification windows (bug 949427)
  • Problem on how we're connecting our slave nodes - javaws. There's an issue on jenkins for linux when the node got disconnected. Henrik looked over multiple solutions, filed more issues against jenkins
    • wrote a little script that can be used for puppet too

Mozmill Tests (Softvision)

  • Fixed:
    • bug 1060254 - Test failure 'Geolocation position is: unavailable' in /testGeolocation/testShareLocation.js
    • bug 678456 - getSuggestions in search.js returns suggestions for the first letter only
    • bug 994102 - Search engines don't work right away after installation
    • bug 1087202 - Test failure "Element.waitForElement(): Element 'Selector: *[label='Test Bookmark 1']'" in /testBookmarks_OpenAndClose/test1.js
    • bug 987565 - Test failure "controller.waitForPageLoad" in testBookmarks_OpenAndClose/test1.js and testBookmarks_OpenAllInTabs/test1.js
    • bug 1097700 - Test failure 'Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFile.remove]' in /testAddons/testInstallUninstallSoftBlocklistedExtension.js
    • bug 689417 - tests/functional/testToolbar/testStopReloadButtons.js doesn't fail when explicitly broken // just closed, this was previously fixed by bug 954156
    • bug 1097682 - Test failure 'Unknown property - identity.mixed_display_loaded' in /testSecurity/testMixedContentPage.js
    • bug 1084333 - Add some more methods and a property to BaseWindow class
    • bug 1079725 - Add a BaseInContentPage class
    • bug 1097095 - Windows XP End of Support Pop-up on all XP production machines
  • WFM:
    • bug 929357 - Timeout failure in between functional testruns
    • 11 fixed bugs last week and another 7 are in progress

TPS (Henrik)

  • Needs update for pulse script to support Pulse Guardian
    • upgrade the pulse script to pulse gardian
  • Hopefully this week the server database will be updated, so the signIn() problem might be gone
    • we still have test failures, but are related to server problems - only related to users, dynamically create accounts on the servers, because on the machine we run tests all are passing
  • Shall we permanently enable the new CI? Current failures are all most likely related to server-side issues
    • Cosmin: we don't get notifications for failures from the old system - this has to be checked
    • Marc: as long as we're not missing something, I'm good with doing the switch

WebRTC (Nils, Syd)

  • Started to use the lab for loop, there's work in progress to be able to run tests, due to missing infrastructure for these different components
  • monitoring failures, might be more infrastructure related than webrtc code

Softvision Updates

Personal Status

For the personal status please check the weekly status updates:

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

If you missed the meeting you can also watch it in our archive.

Action items

  • [ON TRACK] [Henrik] follow up on bug 1083166
  • [NEW] [Andreea] Check in-qa-testsuite flags for proxy/network tests
  • [NEW] [Mihaela] Check with the team from AWSY if they have detected our endurance memory increased consumption
  • [NEW] [Henrik] goals wiki page
  • [NEW] [Henrik] check if we can shutdown the old tps system
  • Chair: Andrei
  • Notes: Teodor
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Automation/Meetings/141117&oldid=1034453"