Breakpad/Status Meetings/2015-06-03

« previous meeting — index – next week » ^create?

Meeting Info

Breakpad status meetings occur on Wed at 11:00am Pacific Time.

Conference numbers:

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200# 

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

AWS

• load testing over weekend
  • little over 2X traffic sent into collectors
  • ES + processors needed some tuning
  • turned out submitters were skewing towards non-release, were attempting to process over 3X normal crash rate

• want to do one last load test
  • getting statsd working first

• data migration is next
  • copy fresh PG DB
  • run special-purpose processor to populate ES

• what order to migrate services
  • it's too big of a job to migrate all apps at once
    • we need to test and make sure they are working, be able to fall back to PHX temporarily if not, etc.
  • some apps like symbolapi have no dependencies and DNS could be switched any time.
  • others (like crash-analysis) depend on data migration being complete (crash-stats)
  • proposal, prep and switch DNS in this order (roughly):

1. symbolapi.mozilla.org (no SSL needed for this service!)
2. crash-stats.mozilla.(org|com)
3. crash-reports.mozilla.(org|com)
4. ... everything else

• what else is left?
  • tracker bug: bug 1118288

DB

ES

Other

Project Updates

Socorro Bug Tracker
this week's bugs

PR Triage

• https://prs.paas.allizom.org/mozilla/socorro

other business

Travel, PTO, etc

• rhelmer half-day Friday this week
• lonnen only around monday, friday next week

Links