Autonag, our bugzilla automatic triage bot and alerting system

From MozillaWiki
Jump to: navigation, search

This page lists all the actions done by the Autonag bot.



Every day hundreds of tickets are opened in Bugzilla which track the tasks, defects and enhancements needed for the development of Firefox and other Mozilla projects. Triaging and priotirizing these bugs are an essential part of our development process and we automate part of this process so as to increase our development turn around and improve the quality of our bugs metadata. The set of scripts we use to improve the quality of our bugs, decrease our triaging time and help Release Management ship better quality software is called Autonag.

Initially created as an alerting system (by email) to our triagers, Autonag rules now also make changes to our bugs metadata on Bugzilla so as to fix inconsistencies.

Rules that change automatically some data in Bugzilla (change a priority, needinfo somebody, close a bug…) are called "Rules with Autofix" and those rules are applied once per hour. For now, security bugs aren't touched.

Rules that do not change data but have other results such as generate a report or send an email are called "Rules without Autofix". Rules without autofix are processed once a day (at 2PM CET).


With autofix

Bug assigned but marked as UNCONFIRMED

Purpose Mitigate an issue in Bugzilla (bugs reported by new users are not tagged as NEW
Action Change the status from UNCONFIRMED to ASSIGNED if there is an assignee
Example Bug 1495908

Regression keyword is missing

Purpose Regression keyword is important to differentiate an actual defect
Action Sets the regression keyword on bugs with the regression-range-wanted keyword is set
Example Bug 1461034

Remove leave-open keyword on closed bug

Purpose Clean up a mismatch in metadata
Action If a bug is closed but the leave-open keyword is still set, remove it
Example Bug 1382185

[meta] in title but not in keywords

Purpose Improve metedata quality
Action Adds the meta keyword if the bug title starts by [META]
Example Bug 1435799

Update Firefox Status flags for bugs reopened during Nightly cycle

Purpose Avoids potential issues for sheriffs and release managers
Action Set firefox-status flag back from fixed to affected
Example Bug 1495962

Bug with no assignee but a patch landed

Purpose Attribute unassigned bug to the developer that fixed it
Action The ASSIGNEE field on the bug is changed from to the author of the patch that landed in mozilla-central
Example Bug 1514338

Close crash bugs with no crashes over 12 weeks

Purpose Reduce the backlog of bugs to check for Release Managers
Action Crash bugs without any crash reports for more than 12 weeks
Example Bug 1470864

Remove stalled keyword on closed bugs

Purpose Fix inconsistency between metadata and bug status
Action If a bug is marked as FIXED and also has a stalled keyword set, the keyword is removed
Example Bug 1491624

Has meta keyword but not [meta] in the bug title

Purpose Having [meta] in the bug title helps with quick search results
Action If a bug has the meta keyword set, [meta] is added to the bug title
Example Bug 1257692

Set a Firefox status flag for beta if there is one for nightly and release

Purpose Fix inconsistency in Firefox status flags that can lead to a bug going undetected by Release Managers during the beta cycle
Action If a status exist for Firefox N-1 and for Firefox N+1, guess a value for Firefox N
Example Bug 1500273

Increase priority of bugs tracked y Release Managers

Purpose Priotitizes bugs needing an action for shipping quality software
Action If a bug is tracked by a Release Manager, update the priority (P2 for nightly, P1 for the rest)
Example Bug 1515946,1508277,1512493
Note There are multiple files mismatch-priority-tracking-*.py, one per channel

Add regression keywords to bugs (uses Machine Learning)

Purpose Surface regressions not filed as such
Action If a bug is tracked by a Release Manager, update the priority (P2 for nightly, P1 for the rest)
Example Bug 1529139

Move untriaged bug into the correct component (uses machine learning)

Purpose Decrease manual triagin time
Action Uses machine learning to mass move bugs in Firefox::Untriaged into a better component
Example Bug 1530316

Copy crash signature from duplicate bugs to main bugs

Purpose Crash bugs marked as duplicate of another one may have a different crash signature. We need to consolidate all signatures in the bug where a patch to fix them is being worked on, other wise we may not fix them all
Action If a crash bug is marked as a duplicate, the signatures it referenced are added to the new bug
Example Bug 1517205

Increase severity of bugs tracked by Release Managers

Purpose Fix inconsistency in metadata
Action If a bug is tracked for an upcoming release but it's severity field is low, the severity is increased
Example Bug 1538966

Nag triage owner to set priority on bugs

Purpose Bugs with no priority set are harder to prioritize for Release Managers with regards to the release
Action The triage owner is needinfoed in Bugzilla or nagged via email to set a priority on bugs
Example Bug 1527818
Code workflow/

Without autofix

Feature vs regression

Purpose Fix inconsistency in bugs with both the regression and feature keywords set
Action No action yet

Inactive bugs with the leave-open keyword set

Purpose Help identify dead/inactive bugs
Action Needinfo the triage owner if a bug is inactive but has the leave-open keyword set
Example Bug 1367072

Inactive Meta bugs

Purpose Help identify dead/inactive bugs
Action Needinfo the triage owner if a meta bug has no activity and no dependencies set

Low-severity bug but tracked by a release manager

Purpose Idenitfy mismatch in metadata, a tracked bug should not have a priotity lower than normal
Action No action yet

Alert for lack of feedback in a bug

Purpose Increase reaction time on bugs prioritized by upper management or release managers
Action If a bug as an unanswered NeedInfo request from a release manager or a director, send a warning email

Bugs with no priority and no activity

Purpose Triage owners need to process the backlog
Action Needinfo the triage owner (only :Overholt for now)
Example Bug 1503461

Bugs with only one or two words in the summary

Purpose Make sure the bug is actually useful, a two-words summary often indicates a poor quality bug report
Action No action yet
Example Bug 1512823

Remind developers about tracked bugs

Purpose Make sure that bugs tracked for the next release are addressed
Action No action yet

Reporter not answering to Needinfo request

Purpose identify bugs stalled because we need more information fron the reporter to reproduce it
Action No action yet

Top crashers vs normal severity

Purpose Consistency and help getting traction on bugs
Action No action yet
Example Bug 1471692

Identify tracked bugs with a NeedInfo request

Purpose Make sure that bugs tracked for a release or nominated for tracking which also have a NeedInfo request do not get stuck
Action Send an email

Tracked bugs untouched for a week

Purpose Identify important bugs for a release which are not being acted upon
Action Send an email

Regressed in upcoming release

Purpose Identify potential regression
Action Send an email with the list of bugs marked as non affecting a release but affecting the next one

Identify unlanded uplifts to Beta and ESR

Purpose Make sure that we ship with the bugs that we need
Action Send an email wih the list of bugs with uplift requests not landed on the affected branch
Example Bug 1509394

Bugs with high severity in Firefox::Untriaged

Purpose Identify potentially important issues left untriaged
Action No action yet

Version set but not status_firefox

Purpose The Version value is set automatically by Bugzilla on landing a patch but not the status_firefoxXX
Action Send an email

Check release dates on the Wiki

Purpose Verify the consistency of our release dates on as we use them in automation
Action Send an email
Code ../

Information about bugs landed during Soft Code Freeze

Purpose List fixed bug with patches which landed in mozilla-central during the soft freeze week. The listing includes patch statistics.
Action Send an email

Hightlight Fixed bugs in nightly we may want to uplift

Purpose Identify potentially upliftable bugs.
Action Send an email with the recent fixes, the affected branches and metadata for prioritization

New user with a Needinfo request

Purpose Help identify potentially unactionnable bugs per lack of information
Action Send an email

Patch not landed

Purpose Identify bugs with an unlanded r+ patch. Maybe the patch is now obsolete or the bug needs to have a sheriff land the code with the checkin-needed keyword set.
Action Send an email