Release Management/FirefoxOS/Release Milestones

⚡ Warning: The content of this page is obsolete and kept for archiving purposes of past processes.

Firefox OS Release Cycle and Repos

Note: Since 2.2, we already increase FL period to 12 weeks. This graphic has to be updated.

A comparison of Firefox and Firefox OS trains:

(SVG source)

FEATURE LANDING (FL)

Feature Landing (FL) is a fixed milestone in the release schedule. All feature work must be complete by the FL date.

Feature Landing Criteria

• QA signoff
  • Passes functional testing necessary to meet user story acceptance criteria
  • Features must not land with device automated tests disabled
  • No smoketest regressions
  • MozTrap test coverage must be added for each roadmap feature in a particular release
• No performance or checkerboarding regressions
• Includes integration/unit tests
• Performance/stability metrics maintained at least on par with previous release

Notes:

• QA and release management should be informed of all complex feature/Patch landings before the landing occurs.
  • Complex features/patches are defined as:
    • features that have a significant amount of risk wrt destabilizing the tree
    • features that touch multiple modules
• Partial landing of a feature may be acceptable
  • Feature scope change must be signed off by stakeholders including product, UX and QA (may also include other teams like security)

What if a feature misses the Feature Landing date?

• By default the feature is de-scoped from the release and deferred to the next release
• Exceptions can be granted based on criteria such as business need, % feature work completed and expected delivery date, and risk of backing out or preffing off.
• 1-2 weeks before the FL date the Engineering Project Mangers (EPMs) will prepare the list of outstanding features and the release manager, release owner, a product rep, and a QA rep will review the outstanding feature list and make a decision about whether each feature that is at risk of missing FL will remain in the target release or be de-scoped
  • Acceptance of a late feature delivery may come with conditions such as a target landing date and restrictions to the number of string changes
  • All features that are to land after FL must first land on master/mozilla-central and have QA sign off on uplift

STRING FREEZE

• String Freeze marks the beginning of the localization cycle, which includes translation work and translation testing.
• The expectation is that string changes will be landed during the development phase. The target is to land ~90% string changes by FL.

What if I need to land string changes after FL?

• A small number of string changes may be acceptable after FL.
• Tag any bug that requires string changes after FL with the late-l10n keyword. These bugs should be tagged as early as possible.
• String Freeze is targeted for two weeks after FL. late-l10n tagged bugs may land in this two week period. There should be no additional string changes after String Freeze has been declared.
• Options if you find that string changes are required after String Freeze:
  • If you have to land minor polish fixes for your feature, land string changes before the String Freeze date and complete your work after the String Freeze date.
  • Reuse existing strings in the product instead of adding new ones.
    • This should be done only after having consulted the l10n team, see why it is tricky to reuse strings
  • If you need to make a string change after the String Freeze date, speak with release management and l10n about options.

FEATURE COMPLETE (FC)

The Feature Complete (FC) milestone marks the hard cut off date for all feature work related to the current release. In order to declare FC, QA must have completed a set level of testing. Declaring FC indicates the build is ready for chipset vendors to start chipset related testing. FC is a flexible milestone with a target date. FC will only be declared when all of the FC criteria have been met.

Feature Complete Criteria

• All feature landing exceptions must be resolved
• Bugs with keyword late-l10n should be zero
• Chipset vendor requirements or bugs needed to meet their FC date must be resolved
• QA sign-off
  • Completed one functional test run
  • QA defines 2 areas of "functional areas" for exploratory testing and lead will send a "signoff" note when he feels its qualitatively ready
  • Smoketest Greenness - Latest length of time that we haven't had smoketest regressions needs to be greater than five days
  • MTBF Test Run lasts for 30+ hours
• Perfomance team sign-off to ensure we have met the FC ready criteria

Notes :

• Hard string freeze happens in two weeks as we enter the the FC phase giving the localization team four weeks to localize the strings and prepare for a testrun at FC. FC will typically mark the launch of the initial l10n test run.

CODE COMPLETE (CC)

This Code Complete (CC) milestone marks the completion of the release from Mozilla and partner chipset vendors. This is our declaration that we are ready to ship. After this milestone no further Mozilla or chipset vendor specific changes will be accepted. At this point all Mozilla wanted blockers should be resolved leading us to zero blockers. This drop dead date indicates that we are ready to declare a Release Candidate (RC) and Original Equipment Manufacturers (OEM) / Original Device Manufacturers (ODM) can pick our release for their test cycle. The following criteria needs to be met.

Code Complete Criteria

• No open issues across all user stories
• Less than 10 valid release blockers. (Additional blockers may be accepted from IOT/certification testing.)
• Achieve major chip-set vendor CS milestone.
• Full QA sign-off
  • Target 3 test runs (2 min, 3 max)
  • Final test run should >= 98% test case completion
  • Test failure % is trending downwards at a healthy level (ie. in 1.3, TR1 = 11%, TR2 = 8%, TR3 = 6%)
  • Smoketest Greenness - Latest length of time that we haven't had smoketest regressions needs to be greater than five days.
  • QA defines 4-5 areas of “functional areas” for exploratory testing and lead will send a “signoff” note when he feels its qualitatively ready.
  • MTBF test run lasts for 100+ hours
• Sign-off from security, stability(MTBF), performance, product
• Support documentation published
• Release notes completed and ready to publish

General Availability (GA)

note: The GA piece is still being worked on with TAM's/partners and should get updated soon

A release is will be declared General Availability (GA) once the OEM/ODM has completed the IOT/certification testing and granted Terminal Acceptance (TA), which implies the build is good to hit silicon.

General Availability Criteria

• Zero blockers for the release - all targeted partner blockers must be fixed.

Note : In general, there can be agreed upon exceptions to the above and Release Management will communicate progress on each milestone on all mailing lists (internal, public, partners) effective 2.0