Releases/Firefox 3.6.4/Test Plan:OOPP Testing

From MozillaWiki
Jump to: navigation, search

OOPP Focused Testing for 3.6.4

We are going to ship OOPP to a wider audience in this branch. By default it will have three specific plugins whitelisted: Flash, Silverlight and Quicktime. These will run out of process in Windows and Linux. Out of process will not be suppported on Mac.

Plugin versions to focus on:

Run the plugins tests in the BFT:

Focus on:

  • Exploratory Testing:
    • End to end crashed-plugin UI
      • Flash 10.x and 10.1rc, Silverlight 3/4, Quicktime 7.6
    • Crashed-plugin UI elements and their properties
    • Quicktime 7.6 testing on Windows (any other version)
    • Bing toolbar (uses Silverlight)
    • Any other popular toolbars (Google, Yahoo)
    • Flashblock or a couple of popular extensions
    • One P1 locale (de, fr, pt-br, it... for example)
    • Popular sites compatibility
    • Top sites automated testing (Tomcat)
  • Performance Testing:
    • Plugin Testing (Quicktime, Flash and Silerlight) in full screen mode
    • Plugin Testing - HD where appropriate or varying the resolution
    • Visual comparison with other browsers in cases where a performance lag is seen

Results Summary

During our testing we saw several new bugs, but the common theme among them is that most of them are hard to reproduce. The majority of the new bugs are OOPP related. Looking at the crash statistics, it seems 3.6.4pre is almost twice as crashy as 3.6.3, but the crashes are different. (We are missing numbers for 3.6.4). In my opinion none of the bugs we found are blockers.

Windows 7 (juanb)

  • Status: Completed (no major issues)
  • Build ID:
  • Plugins Tested
    • Flash 10.0 r45, 10.1 rc
    • Silverlight 3 and (4.0.50401.0)
    • Quicktime 7.6.6
  • Add-ons used
    • Bing Bar 5.0
  • Ran the plugins section of the BFT.
  • New bugs filed during testing:
    • bug 559484 - crash in [@ mozilla::ipc::RPCChannel::..]
    • bug 559943 - crash using Silverlight 4 context menu
  • Existing bugs seen:
    • bug 469267 - crash/hang in [@GCGraphBuilder::NoteXPCOMChild...] (difficult to reproduce)
  • Issues Noted:
    • Unable to reproduce: crash on http://weblogs.mozillazine.org/marcia/archives/plugin-test.html (with Flash, Quicktime, and Java installed).
    • Unable to reproduce: Under certain conditions, with several pages with different plugins open, killing the plugin-container for Flash, and then reloading the page will kill the whole browser.
    • Investigating: Auto resume while watching a Flash video in full screen doesn't seem to refresh the video, but it will keep going. If you re-size to the original it will pick up where it left as far as video rendering. We noticed this with the network latencies we're experiencing yesterday.

Linux (juanb)

  • Status: Completed (no major issues)
  • Build ID: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
  • Plugins tested:
    • Flash 10.0 r45, 10.1 rc
    • Novel Moonlight 2.2 (in process), Silverlight 3.0x
    • QuakeLive Game Launcher 1.0.277
  • Add-ons used:
    • Yahoo Toolbar 2.1.3.20100310105313
  • Ran the plugins section of the BFT
  • New bugs filed during testing:

Windows XP (marcia) spot check

Windows Vista (marcia) spot check

Mac OS X 10.6 (marcia)

  • Status: Completed
  • BuildID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.4) Gecko/20100413 Firefox/3.6.4
  • Plugin Versions tested
    • Shockwave Flash 10.1 r53
    • Silverlight Version: 3.0.50106.0
  • Bugs Seen
    • Bug 510846 Flash Player settings/preferences don't receive clicks/mouse events on certain sites
      • Highly annoying since sites that require webcam access such as tokbox require accessing settings.
  • Issues Noted

Mac OS X 10.5 (marcia)

Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_3.6.4/Test_Plan:OOPP_Testing&oldid=216513"