Releases/Firefox 21.0b6/BuildNotes

From MozillaWiki
Jump to: navigation, search

Summary

  • Who: kmoir
  • Tracking bug: bug 854371
  • bugs encountered
    • bug 811708 race condition in aggregating scheduler

Process

Build 1

  • l10n & ship-it handled by :bajaj
  • release approved by nthomas
  • There was a race condition encountered in the scheduler.
mysql> select * from schedulers where name = 'release-mozilla-beta-linux_repack_complete';
+-------------+--------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| schedulerid | name                                       | class_name                                    | state                                                                                                                                                                                                                                                                                                                                                                                                  |
+-------------+--------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|        3125 | release-mozilla-beta-linux_repack_complete | buildbot.schedulers.basic.Dependent           | {}                                                                                                                                                                                                                                                                                                                                                                                                     | 
|        3865 | release-mozilla-beta-linux_repack_complete | buildbotcustom.scheduler.AggregatingScheduler | {"remainingBuilders": ["release-mozilla-beta-linux_repack_4/6"], "upstreamBuilders": ["release-mozilla-beta-linux_repack_1/6", "release-mozilla-beta-linux_repack_2/6", "release-mozilla-beta-linux_repack_3/6", "release-mozilla-beta-linux_repack_4/6", "release-mozilla-beta-linux_repack_5/6", "release-mozilla-beta-linux_repack_6/6"], "lastReset": 1367377370.3844531, "lastCheck": 1367391427} | 
+-------------+--------------------------------------------+-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.01 sec)

The scheduler still thought linux repack 4 needed finishing although it had completed. Repacks 6,4 and 2 had all completed around the same time. Catlee fixed this via the manhole.

To fix this, manhole into build master, then execute something like this:
>>> schedulers = list(master.scheduler_manager)
>>> for s in schedulers:
...     if s.name == 'release-mozilla-beta-linux_repack_complete':
...         break
...
>>>
>>> s
<buildbotcustom.scheduler.AggregatingScheduler instance at 0x2c662dd0>
>>> state = master.db.runInteractionNow(s.get_state)
>>> state
{u'remainingBuilders': [u'release-mozilla-beta-linux_repack_4/6'],
u'upstreamBuilders': [u'release-mozilla-beta-linux_repack_1/6',
u'release-mozilla-beta-linux_repack_2/6',
u'release-mozilla-beta-linux_repack_3/6',
u'release-mozilla-beta-linux_repack_4/6',
u'release-mozilla-beta-linux_repack_5/6',
u'release-mozilla-beta-linux_repack_6/6'], u'lastReset': 1367377370.384453,
u'lastCheck': 1367391427}
>>> state['remainingBuilders'] = []
>>> master.db.runInteractionNow(s.set_state, state)
>>> state = master.db.runInteractionNow(s.get_state)
>>> state
{u'remainingBuilders': [], u'upstreamBuilders':
[u'release-mozilla-beta-linux_repack_1/6',
u'release-mozilla-beta-linux_repack_2/6',
u'release-mozilla-beta-linux_repack_3/6',
u'release-mozilla-beta-linux_repack_4/6',
u'release-mozilla-beta-linux_repack_5/6',
u'release-mozilla-beta-linux_repack_6/6'], u'lastReset': 1367377370.384453,
u'lastCheck': 1367391427}

Also, the final verification failed due to problems fetching mar urls for three locales on Linux. Log is here http://buildbot-master63.srv.releng.use1.mozilla.com:8001/builders/release-mozilla-beta-final_verification/builds/0/steps/final_verification/logs/stdio

Wed May  1 11:51:25 PDT 2013:  FAILURE: Could not retrieve http header for mar file from http://download.mozilla.org/?product=firefox-21.0b6-complete&os=linux64&lang=zh-CN&force=1
Wed May  1 11:51:25 PDT 2013:  FAILURE: Could not retrieve http header for mar file from http://download.mozilla.org/?product=firefox-21.0b6-complete&os=linux&lang=ast&force=1
Wed May  1 11:51:25 PDT 2013:  FAILURE: Could not retrieve http header for mar file from http://download.mozilla.org/?product=firefox-21.0b6-complete&os=linux&lang=cs&force=1

Erros on http code 302, redirection. I reran the final verification build and it completed successfully.