Previous Action Items

Status

• OF:
  • last week: 6.64
  • 4 weeks ago: 10.08

• high frequency intermittents
  • last week 31
  • 4 weeks ago 55

• if all high frequencies were fixed, OF would be...?
  • 4.75
  • 5.13

• year to date: 198 fixed, 110 disabled (disabled is 35.7% vs 34.9% 1 month ago)

Discussion Topics

• [gbrown] Starting to look at "new/modified test verification" - bug 1357513
  • overall great idea, web-platform does this on upstream new tests, 10x per test
  • keep it small, don't do dependent files, or run randomization/high resource programs in the background
  • how to track the effectiveness of this:
    • whiteboard tags for intermittents related to test changes
    • build a dashboard for recent changes which are 5+ intermittents in a week
• [gbrown] Trying to enable eslint on more tests - bug 1357557
  • will spend a few days hacking on this to see how easy this is to make a big sweep through the tree
  • might be tough to whitelist existing failures or only run on new changes.
• [jmaher] Lifecycle of managing intermittents today
  • 3 sources:
    • overall OF (looking for missing stockwell tags)
    • neglected
    • needswork
    • [gbrown] reduced OF to the last day or two to find most recent troublemakers
  • is there value to retriggering to identify the root cause
• [jmaher] Roadmap for autoclassification in Q2/Q3 ? (how can we track what is auto classified vs not)
  • query the treeherder db for what is autoclassified- should experiment with this and propose an api
  • Show all intermittents, with auto-classify count, for past week: https://activedata.allizom.org/tools/query.html#query_id=QaxkaHye

{
   "from":"treeherder",
   "select":[
       {"aggregate":"count"},
       {
           "name":"is_auto",
           "value":{
               "when":{"in":[
                   {"literal":"autoclassified intermittent"},
                   "failure.notes.failure_classification"
               ]},
               "then":1,
               "else":0
           },
           "aggregate":"sum"
       }
   ],
   "edges":"bugs.bug_id",
   "where":{"and":[
       {"eq":{"failure.notes.failure_classification":"intermittent"}},
       {"gt":{"last_modified":{"date":"today-week"}}},
       {"ne":{"repo.branch.name":"try"}}
   ]},
   "limit":10000
}

New Ideas to investigate

Action Items

• Discussion topics for next time