(January 2006 This is a draft, but it's a complete, yet imperfect, solution to a highly constrained problem. So it is pretty likely to be adopted with only minor changes. Comments to Brendan Eich.)

Problem Statement

• Ship Firefox 2 no later than Q3 2006 and Firefox 3 no later than Q1 2007
  • Shipping earlier is possible, later is to be avoided at all cost
  • Deliver front-end innovation in Firefox 2 while building it on a more stable back-end platform
  • Complete long lead-time, major architectural advances in graphics and content languages in Gecko 1.9 for Firefox 3
• Avoid requiring developers to merge and land changes in two places (trunk and a major branch)
  • Merging means diverging, which implies conflicts
  • Merging by hand is hard, errors are likely
  • Developers may just forget to land in one place, or decide not to
• Avoid deferred "big bang" landings in any place
  • To maximize testing of continuously integrated changes
  • While minimizing regression checkin-window size
• Keep sufficient community QA focus on 1.9 while 1.8 receives "next release" testing priority
  • Trunk stability is required by our release process to "bake" candidate changes for the branch
  • Gecko 1.9 / Firefox 3 schedule will slip if trunk regresses too much during Firefox 2 development

Branching Diagram

Note that 1.8.1 need not branch until Firefox 2 ships, since nothing else is contending for the 1.8 branch.

Note also that Gecko 1.9 has not yet branched as of September 6th 2007 and so this diagram is somewhat out of date. Also note that the commits to the trunk require approval, and the trunk is being frozen on September 6th 2007.

For more information, refer to http://groups.google.com/group/mozilla.dev.planning/browse_thread/thread/f569a23ce266a451/265344ae5291dee0

(Thanks to Asa for this artwork.)

Proposed Solution

• The 1.8 branch has already branched a 1.8.0 sub-branch for patch releases
  • No checkin without approval via bugzilla flag set by drivers
  • These 1.8.0.x releases will ship every 6-8 weeks
  • They will fix severe reliability bugs (security, crash, even compatibility)
  • Number of fixes limited by time available to QA each patch
• Firefox 2 will come from the 1.8 branch
  • The 1.8 branch will eventually branch for Firefox 2 release and further maintenance as 1.8.1
  • This 1.8.1 number matters to content authors who detect Gecko rv: in User-Agent headers.
  • The goal is to fix only bugs that require no Web content changes
    • All crash, dataloss, leak, and security fixes from 1.8.0.x releases
    • Backward compatibility means not breaking old workarounds for 1.8 bugs
  • New Firefox 2 features requiring back end changes should be minimized
    • Or recast in terms of back end extensions that are purely additive
    • Some amount of intentional API breakage is planned (e.g. History and bookmarks APIs)
  • APIs marked @status FROZEN must not be changed on any 1.8.x branch
  • Changing other unfrozen APIs requires drivers approval (details TBD)
• Firefox 3 will develop on the trunk, which is currently in Gecko 1.9 Alpha Planning stage
  • The trunk hosts continuously integrated and tested rearchitecture work
    • Graphics reimplemented on top of Cairo
    • New XUL features and XUL box layout specification/standardization
    • Embedding and XUL App API and implementation unification
    • Layout "reflow" rearchitecture
    • JavaScript1.9 leading to JS2 / ECMA-262 Edition 4.
    • Python for XUL
    • etc. -- see the Gecko 1.9 Roadmap and Roadmap Scratchpad
  • The trunk is never significantly broken for long
    • "Significant" meaning developers can't work or dogfood on the trunk
    • Back-out or rapid fixing upon regression identification is the rule
    • All major rearchitecture must land in well-tested pieces
    • Again, the main purpose of the trunk is to continuously integrate new work, to maximize QA coverage and minimize regression check-in windows
    • Mistakes will be made, but planned landings with back-out options will minimize (not eliminate) distributed discomfort
  • How to land changes once for both places
    • We will extend CVS commitinfo and bonsai to automate synchronization of files between trunk and the 1.8 branch
    • The synchronization can be specified for source repository subtrees such as browser, mail, xpfe, toolkit, extensions/inspector, etc.
    • Exception lists per directory may be supported if necessary
    • Developers are expected to test appropriately to avoid breaking trunk or 1.8 branch
  • All Firefox 2 changes will land on the trunk as well as the 1.8 branch
    • Some Firefox 2 code will need to be #ifdef MOZILLA_1_8_BRANCH to cope with API or bug-compatibility skew between trunk and branch
    • Some Firefox 3 changes will be needed by Gecko 1.9 back end changes (e.g. for XUL box layout standardization) -- these must be #ifndef MOZILLA_1_8_BRANCH
    • The MOZILLA_1_8_BRANCH macro will be defined for C++ and XUL on the branch
    • After Firefox 2 ships, #ifdefs testing this macro will be purged from the trunk
    • We'll use an automatic unifdef program to avoid fat-finger errors

A Dialog, or FAQ

Q: Why not just develop Firefox 2 on the 1.8 branch and leave the trunk alone?

A: We'll end up with a big-bang landing as happened with the Aviary 1.0 branch. Also, we are more likely to end up with a bugfix or change to the branch pulling all of Gecko over from the trunk, as happened during Aviary 1.0. And mainly, we must keep enough community QA on the trunk so that it does not regress badly, jeopardizing Firefox 3.

Q: What's wrong with big-bang landings, and what do you mean by regression checkin window?

A: Study up on libpr0n and Aviary 1.0 branch landings (among others), and search bugzilla for the regression keyword. In any sufficiently complex C/C++ codebase with test coverage achieved mostly through open-source community testing, too often the only way to gain traction on hard bugs is to identify when they first appeared.

Q: Hmm, then why not avoid merging and big bangs altogether, and share the trunk between Firefox 2 and Gecko 1.9 work for as long as possible?

A: We want to test Firefox 2 alphas and nightlies soon, to make sure nothing breaks by moving a front-end feature to the 1.8 branch. Working only on the trunk might lead to Firefox 2 depending on a trunk-only bugfix or feature. We will provide trunk alphas and nightly builds too, of course (with appropriate code names for the executable).

Q: Ok, but I hate #ifdefs. Why not just merge to the trunk when landing on the branch, and update both trunk and branch with Firefox 2 changes?

A: Merging is painful and error-prone. This has two effects:

• Painful: people avoid merging or forget to do it, leading to more bugs and costlier recovery work to resynchronize.
• Error-prone: hand-merging and re-reading cvs diff output is hard on fingers and eyes, leading to yet more bugs.

By keeping all versions present in the tip revision, using #ifdefs, we make the costs of API divergence clear to all. And really, the goal here is not pretty code, it is significant time savings to hundreds of developers and thousands of testers in reduced bugs from avoiding conflicts and merging in favor of #ifdefs.

Q: Shouldn't those who change APIs bear the cost of adding #ifdefs?

A: Yes, and with module owner and peer review of the affected API-client files as usual. No one gets a free lunch on either side of an API, at the expense of the folks on the other side:

• If you change an API on the trunk, you need to keep the 1.8 branch of the #ifdef working (add the #ifdefs as needed, with review).
• If you break an unfrozen API on the branch with drivers' prior approval, you need to keep the trunk working too.

This cost is a tax, and what you tax, you get less of. It should have the effect of minimizing, ordering, and rationalizing API changes so that we don't have too many, and so that no one gets stuck with his neighbor's tax bill. We will be developing a more detailed plan of action for API preservation and evolution, details TBD.

Q: What's an example of API or bug-compatibility skew where #ifdefs might be required in front-end sources?

A: Fixing window targeting to support better tabbed browsing is one example. A hack-fix only for Firefox could be made to the 1.8 branch, to minimize risk. Meanwhile, to avoid duplication of effort and excess bug habitat, we would like to fix window targeting on the trunk by unifying embedding and XUL-app APIs and implementation, which probably means incompatible API changes.

Another example: XUL box layout standardization may break compatibility at a few edge cases, requiring #ifdefs in toolkit or other front-end XUL sources.

Q: What about the Firefox 2 work already under way on the trunk?

A: It can land on the branch at a stability point its owners are happy with, and thereafter be updated by synchronized checkins from one CVS working tree.

Q: When will the CVS commitinfo, bonsai, or other changes be done?

A: We will work hard to get these done now, the week of 12-Dec-2005, and follow up next week to make sure they're ready for prime time.