Auto-tools/Automation Development/Meetings/130408

From MozillaWiki
Jump to: navigation, search

previous meeting | Meetings | next meeting »

Dial in

 # Vidyo:  https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:  650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #         1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:    irc://irc.mozilla.org:6697/#automation

General

  • Attendees:
    • TBD
  • Chair: Mario
  • Notes: Rob

Last Weeks Action Items

  • [DONE] Dave: Email the mailing list about goal status: Gaia UI tests, update testing, new automation rig in MV and documentation
  • [CARRY OVER] Anthony: Ensure any Metro tests are suitably tagged with in-testsuite?
    • Still working with Juan to identify tests based on previous Metro team sprints

Highlights

  • [henrik] The pulsebuildmonitor support has been landed on staging and will be pushed to production after the next beta release
  • [henrik] Our update tests have been caught a good portion of real updater issues in the last couple of weeks. Especially on localized builds.
  • [dave] The FPS tests are passing again now, thanks to Malini for fixing a race condition in the code!
  • [dave] It appears that bug 774892 is no longer present. This should mean Selenium can implement focusmanager.testmode and may be able to run multiple Firefox binaries in parallel
  • New blockers, any updates?
    • [henrik] Still no update yet for the mozqa.com movement (bug 811869) - How to proceed?
    • [henrik] Still a slow process for IT to get ready with our new Mac Minis (bug 853074)
  • You can find all blockers here: http://bit.ly/Tph0NY

Roundtable

  • TBD

Goals Overview

Project Updates

Gaia UI (Dave)

  • Goals progress:
  • TBD
  • Focusing on getting the tests running smoothly on desktop B2G client. Raised several bugs and tracking them via bug 857622
  • [rwood] Continued work on gaia-ui long-running endurance tests (bug 843083)
    • Improving existing test stability (sleeps for idle time, WebAPI callbacks)
    • Integration with Datazilla (submitted to test project; changes to be made on datazilla server side)
    • Investigating memory issue (bug 850803)
  • [rwood] Developing Bluetooth test to pair device with local machine

WebAPI (Rob)

  • [rwood] No updates this week

WebRTC (Henrik)

  • Goals progress:
    • Identified a problem with the PeerConnection object which will silently hide all internal test failures - bug 857765
    • Still on data channel framework (SDP driven datachannel support landed early last week)
    • Helped in getting a couple of our mochitest timeouts fixed

Softvision Updates

Desktop Automation

  • Goals progress: Q1 - achieved, Q2 - to be determined
  • Fixed:
    • bug 820346 - Test failure "TabView is not open." in endurance/testTabView_OpenTabViewWithTabs/test1.js
    • bug 827752 - Create Mozmill test to verify that Flash content respects Firefox's private browsing
    • bug 821007 - mozmill should log request/response bodies of update requests
    • bug 857380 - 21b1 update tests failing with NS_ERROR_UNEXPECTED in nsIFileOutputStream.init
    • bug 855325 - Missing nspr4 module causes jsbridge to fail when creating the network socket
    • bug 854393 - Failure in testAccessKeys/test1.js - "Disconnect Error: Application unexpectedly closed"
    • Mozmill-ci The DirectUpdate testruns are not running on some Windows nodes
  • Blocked bugs:
    • bug 795398 - Mozmill test to check that MD5 hash signatures are no longer accepted - blocked by bug 804952 - Setup virtual host on mozqa.com for Mozmill MD5 hash signatures check
    • bug 763159 - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js
  • Working on:
    • bug 853005 - Update automation script should exit with a non-zero exit code if unable to get update channel
      • Landed for Mozmill 1.5, update mozmill-automation for Mozmill 2.0
    • Mozmill-ci - Log request/response bodies of update requests
      • we can add env variables by editing the nodes in jenkins, working on appending the content in the same log
    • bug 803489 - Software update tests on Windows 8 fail sometimes due to still running copies of Firefox
    • bug 840022 - "The forward button has been made visible for the 1 page"
    • bug 859241 - Update addons.js to enhance the way addonsManager.close() works
    • bug 786306 - Add more logic to the restart tests to skip following test files if a test is failing
    • bug 793705 - Failure in testSearch/testSearchSelection.js | The tab with index '1' has been selected
    • bug 599290 - Update SoftwareUpdateAPI to support complete relocation to the about dialog
    • bug 783484 - Test failure 'Shutdown expected but none detected before end of test ' in /restartTests/testAddons_uninstallExtension/test4.js
    • bug 800708 - Test failure "All downloads have been finished" in /testPrivateBrowsing/testDownloadManagerClosed.js
    • bug 842571 - Update the process start directory (XCurProcD) in software-update.js library
Waiting for review/Feedback/Need info
Bug number Status People Bug Subject
bug 489474 Review Henrik Write automated tests for user generated video controls
bug 800872 Review Henrik Test failure "Window number '1' has been opened" in endurance/testTabbedBrowsing_OpenNewWindow/test1.js
bug 838192 [Pull Request] Review Henrik/Dave Add screenshot functionality to assertion module
bug 839996 Feedback Marco Add event to wait for before importing the default bookmarks through importFromURL function
bug 834163 Help :) Henrik/Dave Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed"
bug 711129 Review Henrik/Dave Mozmill Endurance test for Open & Close a popup window
bug 838192 Review Henrik/Dave Add screenshot functionality to assertion module (low priority with mozmill 2.0 looming)

Mobile (Robocop)

  • Progress on Robocop Automation
  • Bugs with work in progress:
    • bug 820859 - Robocop: Add test for 'Clear Private Data'
    • bug 830834 - Robocop: Add test for 'Show Search Suggestions' feature
    • bug 846340 - Robocop: Add test for 'Clear Site Settings'
    • bug 838596 - Robocop: Add test for 'Master Password' feature
    • bug 846296 - Robocop: Add test for 'Add Search Engine' feature
    • bug 833832 - Robocop: Add test for Import from Android feature
    • bug 855978 - Robocop: upgrade to robotium-solo-4.0.jar
    • bug 836818 - Intermittent testOverscroll | Checking gecko didn't draw unnecessarily - got true, expected false
  • Recently integrated patches:
    • bug 822259 - Robocop: Add test for Find in Page feature
    • bug 820276 - Robocop: Make editBookmark and checkBookmarkEdit methods available to other tests
    • bug 856518 - Robocop: Add better logging messages for the testDoorHanger test
  • Notes/Issues/Questions:
    • Robotium 4.0 seems to be unstable and causing failiures.
    • Still having issues with the Import from Android test - the database file for the Android Stock browser does not seem to be accessible on Tegras from the test harness
    • We are having issues building the latest sources. Field bug 859341 for this issue

Personal Status

For the personal status please check the weekly status updates:

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

Action items

  • [CARRY OVER] Anthony: Ensure any Metro tests are suitably tagged with in-testsuite?