Breakpad/Status Meetings/2017-02-22

From MozillaWiki
Jump to: navigation, search

« previous meeting — index – next week » create?

Meeting Info

Breakpad status meetings occur on Wed at 10:00am Pacific Time.

Conference numbers: 

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200#

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

  • ES cluster size is up 2 nodes in prod
  • our business is uptime
  • business is boomin'
  • ES is great

--

  • load testing environments exist now
    • one that doesn't auto scale
    • one that auto scales

--

  • finishing antennae pipeline work
    • every deploy has its own monitoring instance (newrelic)
    • blue/green deployments

--

-- coming this week: ES5 cluster (maybe) (stretch goals)

Project Updates

Deployment Triage

PR Triage


Major Projects

Splitting out collector (Antenna)

  • (miles, mbrandt, willkg, rpapa) Had a big meeting about load testing needs, plan and schedule.
  • (miles) Set up a -stage-like environment for load tests. Working on redoing deployment pipeline.
  • (mbrandt, rpapa) Working on load testing for the next week hopefully with results by Friday, February 24th.
    • mbrandt is skipping our meeting today, I was able to schedule some time to run tests and analyze results with rpapa for this time slot.

Deprecation rampage

Processor rewrite

Added some new stuff:

  • AddonsRule()
  • DatesAndTimesRule()

Lots still to do:

Next time we get together as a group we will want to whiteboard some things out. So far I'm still on uncontroversial porting of older code.

Upgrading elasticsearch

  • (Adrian) solved the "matrix" issue
  • (Adrian) down to only 3 failing tests, w00t!
  • (Adrian) next step is to have all tests passing and create a mapping file to use during reindexing
  • (Adrian) ES 5.2 went out recently, should we upgrade to it instead of 5.1?

Other Business

  • Crash Reports schema file is not updated on S3: https://bugzilla.mozilla.org/show_bug.cgi?id=1340105
    • that file is pushed to s3 via a crontabber job, however that job runs on a long-running admin node and the code doesn't automatically get updated on it
    • is there a written process for updating that file?
    • are there complexities in updating the code on the admin nodes? is that written down somewhere?
    • how do we want to deal with code deploys and long-running nodes?

--

  • does new relic get frame data?
    • is this a problem with running on antennae
    • lets open a bug about it!

--

  • we're going to try to get the ES5 cluster going (stretch)
  • heading for antennae as prod collector
    • load test could cause an iteration cycle
    • let's try to set up so that if things go well we could ship this Q

Travel, etc

Links

Retrieved from "https://wiki.mozilla.org/index.php?title=Breakpad/Status_Meetings/2017-02-22&oldid=1163574"