SeaMonkey/QA/Triage HowTo

From MozillaWiki
< SeaMonkey‎ | QA
Jump to: navigation, search
Schieblehreetchor.png
SeaMonkey QA
SeaMonkey Homepage
QA Homepage
Activity Areas
Bug Triage Manual
Fix Checkin Tracking
IRC Chat Logs

Triage (from the French verb trier, meaning "to sort") is the name we give to the action of moving new bugs where they belong, including resolving them if it can be done without actual coding. The following is an attempt at describing (from a SeaMonkey viewpoint) how it is done. We aimed at completeness, maybe at the expense of a quick grasp. If you don't want to read this document in full at one session, have a look at the table of contents and start with what interests you most.

Before you start

For QA Newbees

Following these hints will avoid Problems vor you caused by QA tests with Seamonkey!

  • If you are inexperienced in QA, you should consult SeaMonkey Support with your observations before you file a Bug Report on Bugzilla. You can subscribe newsgroup "mozilla.support.seamonkey" at newsserver news.mozilla.org or subscribe SeaMonkey support mailing list with the same contents like Newsgroup.

Before you start testing, create a new UserProfile. Then use it for your tests , especially if you do your tests with later versions than your normal one. And anyway, backup your existing User Profile - you never know! Experience shows that 2.53 will break 2.49 Bookmarks! Later on, you can just reuse that same profile.

All

  • Make sure you have a supported computer and operating system.
  • Make sure you have a Bugzilla account (and if you don't yet have one, get one here). For effective bug triaging you will need canconfirm privilege, you can check your Bugzilla account privileges here and request additional ones here after you have contributed useful comments to several Bugs.
    • If you have a Persona account, you can use it to sign in to Bugzilla. If you don't, you can get one here before you register with Bugzilla.
    • Or you can register with Bugzilla "the old-fashioned way", with an email address and password.
  • Install the latest SeaMonkey nightly but not in the same directory as your usual "release" version of SeaMonkey (unless, of course, it is already "your usual SeaMonkey" or you want to make it such; but beware that such a practice is "not recommended for most users"). How to install it varies depending on your OS.
  • Notes:
    • "Mac" builds are "Universal builds" supporting both 32- and 64-bit Mac computers with Intel processors running Mac OS X 10.5 or later. OS X 10.4 is not supported anymore, and neither are PowerPC computers.
    • There are separate "Linux" builds for 32-bit (i686) and 64-bit (x86_64) systems. 32-bit builds can run on 64-bit systems (if you have the necessary 32-bit libraries installed) but not vice-versa. The 64-bit builds are still regarded as "experimental" because there are still few testers using them: a chicken-and-egg problem. If you have a 64-bit Linux system, maybe you can help us by testing the x86_64 Linux build.
    • For Windows, only 32-bit builds are officially available at this time. They ought to run correctly on both 32- and 64-bit Windows computers.
    • currently, user Bill Gianopoulos wg9s unofficially supports Linux and Windows builds of SeaMonkey, currently versions 2.53, 2.53 de, 2.57 and Trunk. Some user experience you find here.
    • A full list of hardware and software requirements for Windows, Linux and Mac can be found here.
  • Start your new SeaMonkey as follows (at a command-line [shell] prompt, after cd to the directory containing the executable):
    ./seamonkey -no-remote -ProfileManager
    • -no-remote makes sure there will be no interference from any other instance of SeaMonkey which might be already running, or which you might start later.
    • For more details about the Profile Manager, see this Mozillazine KB article and/or this Mozilla Support article. (The latter is written for Firefox but substituting seamonkey where firefox is used ought to work.)

Which bugs to focus on

This table tells you how many bugs are open but not yet ASSIGNED for each SeaMonkey component. Each of the numbers in the table is a link to a list of bugs. Click the Total heading at top right of the table to sort by descending number of bugs: this will bring to the top the Components most in need of attention. Most important in that table are the UNCONFIRMED Bugs, which have not yet been reproduced.

What to do with them

Now what shall we do with all these bugs?

Triage

The goal of bug triage is to move the bug reports to the correct Product and Component, and to add missing or to correct incorrect information in them. This also includes a meaningful Summary and a precise description of the problem in accordance with the Mozilla bug writing guidelines.

Moving to the right Product / Component

Is the bug in the right Component? Or even in the right Product? Bugs which affect SeaMonkey can belong in any of the following Products (clicking each link will give you a list of Components, explaining briefly what each of them is for). Most SeaMonkey bugs go in the Products near the top of this list:

  • SeaMonkey (anything specific to SeaMonkey, and all SeaMonkey frontend bugs other than those for the Addons Manager)
  • MailNews Core (backend bugs affecting both SeaMonkey and Thunderbird)
  • Toolkit (This includes, among others, all Addon Manager bugs, the bugs for the Firefox-style Download Manager but not the Jökulsárlón one, Password Manager bugs, etc.)
  • Core (backend bugs affecting all three of Firefox, Thunderbird and SeaMonkey, especially about HTML page layout)
  • Plugins (i.e. Acrobat, Flash, Java, and the like)
  • Other Applications (among others, ChatZilla and DOM Inspector, both of which are included in official versions of SeaMonkey)
  • Mozilla Localizations (bugs about menus & messages in other languages than en-US)
  • NSS and NSPR (these two change very little, only very few new bugs belong in them)

Resolutions: DUPLICATE

  • Can you find another valid bug report for exactly the same problem? Then one of them is a DUPLICATE of the other and should be RESOLVED as such. In general the older bug is left open and the newer one is closed as its DUPLICATE. The exception is when the newer bug is more detailed and contains more info which can lead to a solution: in that case, a comment should be added to the older bug explaining why it is resolved as a dupe of a newer bug.
    • In most cases you should not select DUPLICATE before both bugs have been reproduced.
    • Always check date and SeaMonkey Version of appearance for the problem. If the reported problems in 2 reports have the same (or at least similar) symptoms, but they definitively appeared in different Versions, they can't be DUPLICATEs, because they have different roots and need different bugfixes.
  • Can you remember seeing another case of the same bug in the recent past, without being able to find it back now? In that case you may add DUPEME to the Whiteboard, but only if you are positive that you saw another bug for the same problem.

Resolutions: INCOMPLETE and WORKSFORME (and NEEDINFO flag)

Is the bug still found in current SeaMonkey versions (latest release, latest beta, latest Aurora nightly, latest comm-central-trunk nightly)? If you can't decide, add a comment to the bug asking the reporter, and in that case click the NEEDINFO checkbox at the bottom of the bug page, set the info provider to "reporter" in the rolldown, and also add [CLOSEME yyyy-mm-dd INCO] or [CLOSEME yyyy-mm-dd WFM] (where yyyy-mm-dd is a "reasonably" future date) to the Whiteboard.

  • Note: If the bug "depends" on some other open bug (as shown near the top of the bug page), it is normal for it to wait for that oher bug to get FIXED. In that case, asking whether it is still found can wait.

Resolutions: INVALID

Is the bug a valid bug? If the behaviour it describes is intentional, then the bug is INVALID. If the problem is not a Mozilla problem (if the fault is in an extension, or in the code for some plugin, etc.) the bug is also INVALID but in this case the problem should be reported to the responsible party (the extension author, the plugin makers, etc.)

  • If the bug is an OS bug (e.g. in X11 or GTK) or a bug in some popular third-party site (Facebook, Google, …) then it is also not a Mozilla bug but coding a temporary workaround might be useful for SeaMonkey users. Ask the developers (as for a WONTFIX, see below).

Resolutions: WONTFIX

Is the bug a real bug or enhancement request, but one not worth fixing? Only an owner or peer of the component where the bug is found can decide to resolve it WONTFIX. Check the SeaMonkey project areas page and/or the Mozilla module owners list to see who is in charge.

Bug fixing

  • Finally, do you think you can fix the bug? Don't panic!. Some of them, including very easy one-liners, will have the good-first-bug keyword. Some will also have [lang= (meaning the programming language) and [mentor= (with the IRC nick of someone to ask help from) in the Whiteboard; and even if you aren't yet capable of fixing any single bug, all the above will still need to be done.

Resolving bugs

Here are the main Resolutions you will probably use. (There are others.) Most of the bullet headings can be clicked to scroll to a fuller explanation under Triage above.

  • DUPLICATE — This bug is a duplicate of some other bug, whose bug number must be mentioned.
  • INCOMPLETE — The steps to reproduce are either nonexistent or too vague to allow reproducing the bug, and the reporter hasn't replied to requests for further information.
  • WORKSFORME — All attempts to reproduce the bug have failed; or the bug used to be found in earlier versions but it has disappeared from current versions, and we don't know what fixed it. In the latter case you should set Target Milestone ot the earliest known version for which the problem is no longer reproducible; this helps to distinguish the problem from other ones with similar symptoms, but different causes.
  • INVALID — This is either intended behaviour ("not a bug") or else it is a bug but not in Mozilla code ("not our bug")
  • FIXED — The bug has been fixed in trunk code, and we know exactly which patch fixed it. (See also DUPLICATE above.)
    • In that case, the changeset URL should be mentioned in a comment (Bugzilla can linkify expressions such as comm-central changeset 147a258b369c) and the Target Milestone in the bug heading should be set to the branch where the bug was fixed (normally the branch which was "trunk" at the time of fixing, or in some cases the branch to which the fix was ported).
  • WONTFIX — Only an Owner or Peer for the relevant Module may set this Resolution, see below.
  • Also, the VERIFIED status means that someone other than whoever set RESOLVED has checked that the Resolution is correct. For FIXED bugs, this means in principle that someone (not necessarily a single person) has verified that in recent builds, the bug has disappeared from all three of Windows, Linux and Mac, or at least from those of the three where the bug had been seen before.

Whiteboard entries

One of the entries below may be added to the Whiteboard if it is not clear that the bug should be RESOLVED immediately ("yyyy-mm-dd", where present, should be replaced by some "reasonably" future date).

  • DUPEME — you remember seeing a bug somewhere, of which this bug is a duplicate, but you can't find the other bug back.
  • [CLOSEME yyyy-mm-dd INCO] — bugs to be RESOLVED INCOMPLETE if the reporter doesn't answer.
  • [CLOSEME yyyy-mm-dd WFM] — bugs to be RESOLVED WORKSFORME unless someone can reproduce them (and say how, or at least say that it appears sporadically in such-and-such circumstances)
  • [CLOSEME INVA/WONT?] — bugs which might possibly be one of the following, but you aren't certain:
    • Intended behaviour (INVALID)
    • Not a Mozilla bug (INVALID)
    • Not worth fixing (WONTFIX)
    • Notes:
      • 1. As said earlier, only an owner or peer of the concerned module may set WONTFIX so you may want to bring such a bug to their attention. To find who they are, follow the following links for SeaMonkey in particular or for Mozilla code in general.
      • 2. Some developers have a "Bugzilla address of record" different from their listed email address. Make sure that you have JavaScript enabled (at least on bugzilla.mozilla.org), and use the Bugzilla autocomplete feature in the Cc field of the bug.

Tracking flags

Here we'll talk only about the "status-seamonkey-<something>" flags. These are on the right side of the bug page heading, lower than the Cc list. They are used to track in which versions of SeaMonkey the bug has been seen, and also on which branch it has been fixed. (RESOLVED FIXED means the bug is fixed on trunk, even if the fix hasn't yet been ported to the branches.)

Of course, you need to click (edit) in the Tracking Flags section in order to modify them. It is possible to set these and other flags while reporting a new bug, by clicking the Set bug flags button a couple of lines above the Submit Bug button near the bottom of the page.

Here is what their possible settings mean:

  • unaffected : the bug never occurred in this version.
  • affected : the bug happens in the latest builds for this version.
  • fixed : the bug has been fixed in this version.
  • verified : someone has checked that the fix works for this version in all affected platforms.
  • wontfix : the fix won't be ported to this version. In particular, anything involving string changes will never be landed in a branch other than by the sesquimestrial repository merge. Also, only no-risk fixes will be ported to the Beta branch and only stability & security fixes will be ported to the Release branch.
  • --- (default): none of the above, or unknown, or unspecified.
Retrieved from "https://wiki.mozilla.org/index.php?title=SeaMonkey/QA/Triage_HowTo&oldid=1226776"