Previous Action Items

• jmaher to help kmoir with the failing pandas
• kmoir to send blassey an email with specific bugs they could use dev eyes on
• joduinn to contact apc about ability to keep up with demand.
• gbrown to look into m2 and M8 failures: bug 792873 and bug bug 804885

Status reports

Dev team

Rel Eng

Tegra

• Possible problem with image DCOps has used lately, investigating (bug 808437)
• Discussion on "when" to DNR a tegra needs more participation. Will drive some more.
• Working on getting Reader Mode tests set up bug 796068 (as a subset of non-releng bug 786638

Panda

• running Android 4.0 panda jobs on Cedar:
  • https://tbpl.mozilla.org/?tree=Cedar&noignore=1&jobname=Android%204.0
  • lots of oranges/red failures
• deployed this morning some fixes from Callek and jmaher to address reboot issues

Other

• x86 onhold for now bug 750366

IT

• Pandas - Android
  • Developing a remote re-imaging process for pandas bug 764534 - Done
  • 48 Production ready Pandas bug 800038 - all imaging done, RelEng working on these boards, their tracker bug 799698

• Pandas - b2g
  • Image 12 Pandas by hand for b2g (Chassis 3) bug 797629 - Cards imaged, Done

• Other Updates
  • Tracker bug - 800 production ready panda boards bug 799922
    • 23 Panda chassis are in SCL1. We have several racked however the screws for the rail kits did not ship, so we are waiting for those to rack the others.
    • These 23 chassis house 264 panda boards.
    • A-Team has instructed IT to install the android image on these boards. This does not necessarily mean that all 264 boards will remain imaged for android, IT will reimage as instructed.
  • Order parts for 73 Panda Chassis bug 799808 - All parts are in but 49 power supplies. We have bought alternative power supplies, however they are not enclosed so will need to be replaced when we can get appropriate power supplies.
  • Working on a higher density chassis bug 777393 - On hold due to recent influx of Panda activity

A Team

• 10.56% total failure rate
  • if we removed the crashes that occur frequently in m1, m2, m8 we would be at 5.12% total failure rate

• panda board hacking
  • bug 809364 - [r?] discovered zeroconf is causing sutagent to spin cpu cycles (35%+ total panda cpu). This is only in the racked pandas, not at home.
    • why is the environment so different there? I thought this was related to the long dns names, but something changed between when I ran smoketests in September and what we have now.
  • bug 809366 - [r?] discovered that watcher can get stuck in a loop trying to query/launch sutagent. This ends up spinning really fast in calling isProcessAlive(SutAgent) and eating 30-40% cpu on the panda
  • bug 809012 - pandas would not reboot when sutagent is hung, fixed reboot.py script and this should allow us to self recover better.
  • bug 809367 - pandas seem to get into a state where the sutagent will not execute commands
  • bug 808398 - I continue to see pandas that do not come online after a soft reboot. Before reboot I can ping and/or connect via sut, after reboot [up to 5 minute wait] I cannot even ping the device whereas normal behavior is pinging within 30 seconds of hard reboot.
  • bug 809534 - [r+] add support for versioning to watcher

Autophone

• bc has had some progress getting robocop running on real phones.

Eideticker

• No updates. Been entirely focused on B2G.

Round Table

Action Items

• [carried] joduinn to contact apc about ability to keep up with demand.
• Melissa to get a list of network changes for joel