Auto-tools/Automation Development/Meetings/130225

From MozillaWiki
Jump to: navigation, search

previous meeting | Meetings | next meeting »

Dial in

 # Vidyo:  https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:  650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #         1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:    irc://irc.mozilla.org:6697/#automation

General

  • Attendees:
    • Andreea, Daniela, Mario,Otilia, Andrei, Rob, Dave, Henrik, Joel, Adrian, Clint, Anthony, Mihaela
  • Chair: Andreea Matei
  • Notes: Andrei Eftimie

Last Weeks Action Items

  • [DONE] [davehunt] Follow up with Henrik about what test/data we want to report to tbpl[2]
  • [NEW] Joel will talk to Tony and will look into what robocop tests we could add for performance/stress - currently covering more of the basic functionality is more important
  • [NEW] Adrian will get access to the tryserver - Joel will need to log a bug for that
  • [DONE] [davehunt] - Fix the exception that occurs in mozdownload
  • [DONE] [davehunt] - Revisit the automatic retry of downloads in mozdownload

Highlights

  • New Mozmill release: v1.5.21 that fixed a few bugs. We're green now!
  • Mozmill add-on has been removed from AMO (https://groups.google.com/forum/?fromgroups=#!topic/mozmill-dev/02HwCHWQFRA)
  • MemChaser 0.5.2 has been released
  • Nightly Tester Tools 3.4 has been released
  • mozdownload 1.6 has been released (now with automatic retries)
  • gaiatest 0.8 has been released
  • b2gpopulate 0.5 has been released
  • Gaia UI tests much more stable now that we're restarting b2g between tests (down to three tests failing on master)
  • b2gperf 0.1.3 has been released
  • Endurance tests now have a 5 second delay (increased from 0.1 seconds) to allow memory allocation to settle in a more realistic way
  • New blockers, any updates?
    • bug 838786 - Gaia UI tests suffering from socket timeouts and Marionette status=500 errors
    • bug 840929 - B2G performance tests are failing and unable to recover from failure
  • You can find all blockers here: http://bit.ly/Tph0NY

Roundtable

Add here any issues or topics you want to discuss with the team

  • [Dave] What do we think of the new meeting format? Any other suggestions?
  • [Dave] 'Ask an Expert' is a success! Should we make it twice a week?
  • Mozmill styleguide - Clarify trailing new lines, and how strongly should we enforce this?
    • [davehunt] I suggest specify a single newline at the end of each file in the style guide, but as long as there's at least one new line (and not excessive new lines) we shouldn't block a patch on just this infraction.
    • [Andrei] Styleguide changes when fixing issues. We shouldn't mix them. If we write new code, adhere to the styleguide, but don't random fix style issues when fixing a bug, it leads to less comprehensible code changes.
  • [Andreea] - AutomationBot from #automation to announce when a bug has been fixed
    • We do get this info on emails updates from Bugzilla, but for my email account at least, when I'm connected to the VPN I don't get any emails until I switch back. IRC on the other hand works (after restart).
  • [henrik] Should we add support for different applications in the current Mozmill Dashboard or wait for the new one? We would need this for the Firefox Metro mode
  • [henrik] We need new test candidates for Mozmill. Please get back to the QA team and look at moztrap.

Goals Overview

Project Updates

Gaia UI (Dave)

  • Goals progress:
    • [rwood] Goal: Write stability/long running/endurance tests and graph their performance over time [ON TRACK]
  • [rwood] Developing long-running GaiaUI tests (bug 843083). Current tests under development are for airplane mode, SMS, and music playback. For a complete list please see the GaiaUI Stress Test etherpad.
  • [davehunt] UI tests got a green build last week! Now focus is on v1-train and close to green there too

WebAPI (Rob)

  • Goals progress:
    • [rwood] This goal was completed last quarter except for two tests still under review/need minor updates
  • [rwood] Minor changes required for two Screen Orientation tests, once submitted and they receive an r+ this section can be removed from the meeting agenda.

WebRTC (Henrik)

  • Goals progress:
    • No movement so far
  • Waiting for todays meeting to figure out how the refactor for the test framework will happen. Will continue now with data channel tests.

Softvision Updates

Desktop Automation

  • Goals progress: Getting started, Andreea will work on a test this sprint: test_clock_switch_clock_type.py or test_clock_turn_on_off_alarm.py, as Dave suggested.
  • Fixed:
    • bug 842939 - Testing on all platforms the new mozmill version
    • bug 783484 - Test failure 'Shutdown expected but none detected before end of test ' in /restartTests/testAddons_uninstallExtension/test4.js
    • bug 831686 - Create Mozmill test for proxy.pac file served via a file:// protocol
    • bug 840485 - Test failure "The crash reporter executable is present - got 'false'" in /testInstallation/testBreakpadInstalled.js
    • bug 841355 - Test failure "Password notification should be closed" in some functional tests that use notification.getNode().state
    • bug 800708 - Test failure "All downloads have been finished" in /testPrivateBrowsing/testDownloadManagerClosed.js
    • bug 780957 - Failure in /testAddons_pluginDisabledAfterRestart/test2.js | Shockwave Flash is enabled - 'false' should equal 'true'
    • bug 611455 - 'Application disconnected' failure in restartTests/TestMasterPassword/test.2
  • Blocked bugs:
    • bug 795398 - Mozmill test to check that MD5 hash signatures are no longer accepted - blocked by bug 804952 - Setup virtual host on mozqa.com for Mozmill MD5 hash signatures check
    • bug 763159 - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js
  • Waiting for review / feedback / Checkin needed
    • bug 831406 - Land Flash private browsing tests in testcase-data repository
    • bug 786306 - Add more logic to the restart tests to skip following test files if a test is failing
    • bug 823064 - Refactor testDownloadManagerClosed.js for the new PB per-window mode
    • bug 826251 - Make testPreferences/testPreferredLanguage.js language independend
  • Working on:
    • bug 803492 - Enhance the constructor of MozMillController to set focus on the window
    • bug 489474 - Write automated tests for user generated video controls
      • each platforms requires different values and timeouts, patch almost ready
    • bug 800872 - Test failure "Window number '1' has been opened" in endurance//testTabbedBrowsing_OpenNewWindow/test1.js
    • bug 843553 - Test failure "Checkbox ID: acceptCookies could not be checked/unchecked" in /testPreferences/testDisableCookies.js
    • bug 732353 - Disable all Discovery Pane tests due to unpredictable web dependencies
    • bug 809124 - restartTests/testAddons started failing with "Modal dialog has been found and processed"
    • bug 840022 - Test failure "The forward button has been made visible for the 1 page" in testToolbars/testBackForwardButtons.js
    • bug 842571 - Update the process start directory (XCurProcD) in software-update.js library
    • bug 838192 - Add screenshot functionality to assertion module
    • bug 820346 - Test failure "TabView is not open." in endurance/testTabView_OpenTabViewWithTabs/test1.js
    • bug 707663 - Timeout failure in /testTabbedBrowsing_PinAndUnpinAppTab/test1.js | Tab has scrolled into view

Mobile (Robocop)

  • Automation testing status
  • Progress on Robocop Automation
  • Bugs with work in progress:
    • bug 820276 - Robocop: Make editBookmark and checkBookmarkEdit methods available to other tests
    • bug 822259 - Robocop: Add test for Find in Page feature - blocked by bug 828481
    • bug 820859 - Robocop: Add test for 'Clear Private Data'
    • bug 830834 - Robocop: Add test for 'Show Search Suggestions' feature
    • bug 833832 - Robocop: Add test for Import from Android feature
  • Bugs recently integrated
    • bug 841377 - Robocop: Extend the testShareLink test to cover opening the share popup from context menus
  • Issues/Questions
    • Progress was slow last week on automation and will continue to be this week because of manual testing tasks - Firefox Mobile 19 RC, Firefox Mobile 20 beta 1 and 2, Aurora 21 sign-off
    • The team has also been looking into BFTs features to automate (extend web context menu test to cover images, add search engines and search works on unvisited bookmarks - bugs will be logged later this week)

Personal Status

For the personal status please check the weekly status updates:

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

New action items

  • [CARRY OVER] Joel will talk to Tony and will look into what robocop tests we could add for performance/stress - currently covering more of the basic functionality is more important (email dialog started)
  • [CARRY OVER] Adrian will get access to the tryserver - Jeol will need to log a bug for that (bug 844892 filed)
  • [NEW] [Henrik] - Disable flash on OSX 10.6
  • [NEW] [Henrik] - Start a thread in the mailing list for additional 'Ask an Expert' session
  • [NEW] [Andrei] - amed our style guide to include a section about Bug Fixes vs Style changes
  • [NEW] [Andreea] - File a bug against Automation bot to add the functionality
  • [NEW] [Andreea] - Look into VPN issues (unable to recieve email while connected)
  • [NEW] [Henrik] - Extra item to the issue for mozmill-dashboard. adding support for Metro tests
Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/Automation_Development/Meetings/130225&oldid=534477"