Jump to: navigation, search

Difference between revisions of "Platform UI/Bug Triage"

(Helpful links)
 
(8 intermediate revisions by 3 users not shown)
Line 1: Line 1:
=== Problem ===
+
{{Platform UI}}
 +
= Helpful links =
 +
* [https://bugs.eclipse.org/bugs/report.cgi?x_axis_field=bug_severity&y_axis_field=assigned_to&z_axis_field=priority&query_format=report-table&short_desc_type=allwordssubstr&short_desc=&classification=Eclipse&product=Platform&component=IDE&component=UI&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&format=table&action=wrap&field0-0-0=noop&type0-0-0=noop&value0-0-0= bug overview] (all bugs in Platform UI and IDE, by owners, severities and priorities)
 +
* [http://www.eclipse.org/eclipse/platform-ui/greasemonkey/ Greasemonkey script]
 +
* [http://www.eclipse.org/eclipse/platform-ui/componentAreas.php Component Areas] - We use component area tags in square brackets in the bug summary. The tags facilitate querying of all bugs in a certain component category. For each component area, there is one committer who watches the bugs in that component area. Historically, the committer would be assigned as the owner of the bug. Over time, we will move all bugs that are not worked on or planned to the account platform-ui-triaged@eclipse.org and use the "QA Contact" field instead for purposes of watching the bugs.
 +
* [[Platform_UI/Bug_Triage_Change_2009|Why we changed our bug triage process]] in February 2009
 +
* [http://wiki.eclipse.org/How_to_report_a_deadlock How to Report a Deadlock]
 +
* [[Platform_UI/Bug_Triage_Tips |Bug Triage Tips]]
  
In the current Platform UI bug triaging process, bugs are assigned to component owners.  The primary role of the component owner is to watch over the activity in those bugs, ensuring that no critical bugs are left unnoticed and that comments are addressed.  Second to that is to attempt to fix some of the bugs.  This assigning of bugs is effected through the changing of the "owner" field of the bug.
+
=The New Bug Triage Process in Detail=
  
However, this process incorrectly provides the impression that all bugs are being worked on because they have owners.  This often isn't the case, since the reality is that there are more bugs per component owner than the owner will ever have time to fix.  It makes it look like "everything is being taken care of".
+
[[Platform UI/Bug Triage Change 2009]] - the problem description and transition information
  
Additional tricks like adding the "helpwanted" keyword or changing the priority are then required, but these aren't always or consistently applied, so become an unreliable reflection of the need/desire for community participation.
+
This is how our bug triage process looks like:
 
+
Finally, while being the owner of a component area is a fairly "big" job, watching over bugs and helping with their further triaging isn't, and we recognize that there are opportunities for the community to participate at a lower granularity than component owner.
+
 
+
=== Proposal ===
+
 
+
We believe that it would be clearer, and more helpful to the community, if the assigning of a bug meant what it should mean, which is that the person "owns" the outcome of the bug and is going to fix it.  When and only when a committer decides they are going to fix a bug should it's owner field be set to them.  Bugs without owners are, by definition then, bugs not being worked on.
+
 
+
To manage the "watching" of bugs task, we propose instead to use the QA Contact field, which presently is unused. Where today we'd assign the bug to the component owner, we'd instead make them the QA Contact.  Triaged bugs would initially be assigned to a group user like platform-ui-triaged@eclipse.org.  The QA Contact could be the component owner, or could be any other platform UI committer who is willing to lend a hand.  They would keep up on the comment thread and ensure that critical bugs are brought to the attention of someone who can fix it.
+
 
+
=== Advantages ===
+
 
+
We believe there are several advantages to this small change:
+
 
+
# Clarity of status: It provides a more truthful reflection of bug activity. It now becomes easier to tell which bugs are actually being worked on, since they are the ones with owners.
+
# Identification of opportunity: It provides an easier way for the community to identify places where they can get involved.  Bugs without an owner are defacto "help wanted" bugs.  Additionally, committers who want to help but can't commit to owning a component can become QA Contacts for some set of bugs.
+
# Effectiveness: Finally, we believe this will help component owners focus on fixing the most important bugs, which after all, is the greatest benefit to the community.
+
 
+
===The Process in Detail===
+
 
+
This is how the bug process would look like as a result:
+
  
 
Incoming bugs are triaged by a full-time committer on a rotating basis. The goal is to have an empty inbox, no bug should stay in the inbox for longer than a few days.
 
Incoming bugs are triaged by a full-time committer on a rotating basis. The goal is to have an empty inbox, no bug should stay in the inbox for longer than a few days.
  
Incoming bugs: To do triage, we use the Greasemonkey script (with buttons for all component areas). Clicking on a button will
+
'''Incoming bugs''': To do triage, we use the Greasemonkey script (with buttons for all component areas). Clicking on a button will
 
* Prefix the bug description with [ComponentArea].
 
* Prefix the bug description with [ComponentArea].
 
* Set the "QA contact" field to the person responsible for watching bugs in the identified component.
 
* Set the "QA contact" field to the person responsible for watching bugs in the identified component.
Line 35: Line 22:
 
As part of the triage, we need to make sure that the bug contains enough information, and that the "severity" is set to a value that makes sense. Enhancement request need to be marked as such, and the same is true for blocker/critical bugs.
 
As part of the triage, we need to make sure that the bug contains enough information, and that the "severity" is set to a value that makes sense. Enhancement request need to be marked as such, and the same is true for blocker/critical bugs.
  
When a person is assigned as the QA contact for a bug:
+
When a person is '''assigned as the QA contact''' for a bug:
 
* Double-check that there is enough information in the bug, and that the severity is accurate.
 
* Double-check that there is enough information in the bug, and that the severity is accurate.
 
* If it is a blocker or critical bug (e.g. loss of data, crash), it needs to be fixed as soon as possible. If you are not able to fix the bug yourself, assign it to someone else who is.
 
* If it is a blocker or critical bug (e.g. loss of data, crash), it needs to be fixed as soon as possible. If you are not able to fix the bug yourself, assign it to someone else who is.
Line 41: Line 28:
 
* For all other bugs, try to stay "on top" of the bugs you are watching in the sense that you can produce a rough prioritization of the bugs in a component area. Something like: "if we had the time to fix these ten bugs, our code would be in much better shape".
 
* For all other bugs, try to stay "on top" of the bugs you are watching in the sense that you can produce a rough prioritization of the bugs in a component area. Something like: "if we had the time to fix these ten bugs, our code would be in much better shape".
  
When a new comment is made on a bug you are watching:
+
When a '''new comment''' is made on a bug you are watching:
 
* Respond on the bug if necessary.
 
* Respond on the bug if necessary.
 
* Adjust prioritization if necessary.
 
* Adjust prioritization if necessary.
  
Questions and comments are of course welcome. Unless we find a flaw in the proposed process, or come up with a better idea, we should be able to use the new process next week. We suggest that we start using this new process on all incoming bugs, and move over existing bugs over time. For existing bugs, we should start with those that are currently "owned" by former members of the team (Kim, Tod, Karice...). As part of the moving over, we should classify bugs correctly as actual bugs vs. enhancement requests.
+
=Categorization=
 +
 
 +
==Real==
 +
 
 +
The bug is real and is one we intend on fixing in the milestone specified. Although we cannot promise to address it in this timeframe, this is our intention.
 +
 
 +
Status: '''ASSIGNED'''
 +
Milestone: '''something reasonable'''
 +
 
 +
==Real No Time==
 +
 
 +
The bug is real but we do not have the time to address it. We would entertain patches for these issues.
 +
 
 +
Status: '''ASSIGNED'''
 +
Milestone: '''no milestone'''
 +
Keywords: '''helpwanted if appropriate'''
 +
 
 +
==Real Not A Priority==
 +
 
 +
The bug is real but we do not feel it's that important.  We would entertain patches for these issues.
 +
 
 +
Status: '''ASSIGNED'''
 +
Keywords: '''helpwanted if appropriate'''
 +
Priority: '''P5'''
 +
 
 +
==Not Real==
 +
 
 +
The bug is not valid.  We will not address it further.  If it's a bug regarding established behaviour we do not wish to change mark as '''WONTFIX'''.  If it's simply not a problem as described in the report use '''INVALID'''.
 +
 
 +
Status: '''WONTFIX or INVALID'''
 +
 
 +
=Accepting Patches From The Community=
 +
The following steps should be taken when committing patches from members of the community who are not committers on the Eclipse top-level project.
 +
 
 +
# examine the patch and apply if it's deemed acceptable
 +
# ensure that the bug is assigned to yourself
 +
# ensure that the bug has the "contributed" keyword and mark the attachment as "iplog"
 +
# mark the bug as fixed and include in the comment "patch applied"
 +
 
 +
For more details, see [[Development Resources/Automatic IP Log#Contributors]].
 +
 
 +
John Arthorne had said the following in an [http://dev.eclipse.org/mhonarc/lists/eclipse-dev/msg08253.html email] to the community:
 +
 
 +
''There has been some confusion about the new "iplog" flag so I should clarify. The "iplog" flag should be set on the *attachment* that contains the contribution. This is done by clicking "Details" next to the attachment, and then setting the "iplog" flag for that attachment. If there are multiple attachments being contributed, add the flag for each applicable attachment. This allows the system to automatically harvest data about the size of the contribution, who contributed it (the person who added the attachment), and the committer who processed the contribution (the person who set the "iplog+" flag).''
 +
 
 +
''There is also an "iplog" flag for the entire bug report next to the "review" and "pmc_approved" flags.  This flag should only be used if the contribution is not in the form of an attachment. For example, you can use this if the contributor added code or pseudo-code directly in the comment field. The simple solution in this case is to ask the contributor to attach a proper patch so it can be flagged. If this is not possible or the contributor doesn't respond, you can resort to setting the global "iplog" flag on the bug itself. What this flag does is flag every non-committer who added a comment to the bug as contributors. If there are comments in the same bug that are not part of the contribution, they then need to be excluded manually by editing the project IP log. See the IP log documentation for more details.''
  
TODO: Clean up page after we have consensus. Add links to bugzilla queries, component areas, and greasemonkey script.
+
[[Category:Platform UI]]

Latest revision as of 12:46, 29 July 2009

Helpful links

  • bug overview (all bugs in Platform UI and IDE, by owners, severities and priorities)
  • Greasemonkey script
  • Component Areas - We use component area tags in square brackets in the bug summary. The tags facilitate querying of all bugs in a certain component category. For each component area, there is one committer who watches the bugs in that component area. Historically, the committer would be assigned as the owner of the bug. Over time, we will move all bugs that are not worked on or planned to the account platform-ui-triaged@eclipse.org and use the "QA Contact" field instead for purposes of watching the bugs.
  • Why we changed our bug triage process in February 2009
  • How to Report a Deadlock
  • Bug Triage Tips

The New Bug Triage Process in Detail

Platform UI/Bug Triage Change 2009 - the problem description and transition information

This is how our bug triage process looks like:

Incoming bugs are triaged by a full-time committer on a rotating basis. The goal is to have an empty inbox, no bug should stay in the inbox for longer than a few days.

Incoming bugs: To do triage, we use the Greasemonkey script (with buttons for all component areas). Clicking on a button will

  • Prefix the bug description with [ComponentArea].
  • Set the "QA contact" field to the person responsible for watching bugs in the identified component.
  • Assign the bug to platform-ui-triaged@eclipse.org (separate account from platform-ui-inbox).

As part of the triage, we need to make sure that the bug contains enough information, and that the "severity" is set to a value that makes sense. Enhancement request need to be marked as such, and the same is true for blocker/critical bugs.

When a person is assigned as the QA contact for a bug:

  • Double-check that there is enough information in the bug, and that the severity is accurate.
  • If it is a blocker or critical bug (e.g. loss of data, crash), it needs to be fixed as soon as possible. If you are not able to fix the bug yourself, assign it to someone else who is.
  • If it is a regression, it needs to be fixed during the current development cycle, or potentially in a maintenance stream as well. Schedule the bug accordingly, or assign it to someone else who has time to work on it.
  • For all other bugs, try to stay "on top" of the bugs you are watching in the sense that you can produce a rough prioritization of the bugs in a component area. Something like: "if we had the time to fix these ten bugs, our code would be in much better shape".

When a new comment is made on a bug you are watching:

  • Respond on the bug if necessary.
  • Adjust prioritization if necessary.

Categorization

Real

The bug is real and is one we intend on fixing in the milestone specified. Although we cannot promise to address it in this timeframe, this is our intention.

Status: ASSIGNED
Milestone: something reasonable

Real No Time

The bug is real but we do not have the time to address it. We would entertain patches for these issues.

Status: ASSIGNED
Milestone: no milestone
Keywords: helpwanted if appropriate

Real Not A Priority

The bug is real but we do not feel it's that important. We would entertain patches for these issues.

Status: ASSIGNED
Keywords: helpwanted if appropriate
Priority: P5

Not Real

The bug is not valid. We will not address it further. If it's a bug regarding established behaviour we do not wish to change mark as WONTFIX. If it's simply not a problem as described in the report use INVALID.

Status: WONTFIX or INVALID

Accepting Patches From The Community

The following steps should be taken when committing patches from members of the community who are not committers on the Eclipse top-level project.

  1. examine the patch and apply if it's deemed acceptable
  2. ensure that the bug is assigned to yourself
  3. ensure that the bug has the "contributed" keyword and mark the attachment as "iplog"
  4. mark the bug as fixed and include in the comment "patch applied"

For more details, see Development Resources/Automatic IP Log#Contributors.

John Arthorne had said the following in an email to the community:

There has been some confusion about the new "iplog" flag so I should clarify. The "iplog" flag should be set on the *attachment* that contains the contribution. This is done by clicking "Details" next to the attachment, and then setting the "iplog" flag for that attachment. If there are multiple attachments being contributed, add the flag for each applicable attachment. This allows the system to automatically harvest data about the size of the contribution, who contributed it (the person who added the attachment), and the committer who processed the contribution (the person who set the "iplog+" flag).

There is also an "iplog" flag for the entire bug report next to the "review" and "pmc_approved" flags. This flag should only be used if the contribution is not in the form of an attachment. For example, you can use this if the contributor added code or pseudo-code directly in the comment field. The simple solution in this case is to ask the contributor to attach a proper patch so it can be flagged. If this is not possible or the contributor doesn't respond, you can resort to setting the global "iplog" flag on the bug itself. What this flag does is flag every non-committer who added a comment to the bug as contributors. If there are comments in the same bug that are not part of the contribution, they then need to be excluded manually by editing the project IP log. See the IP log documentation for more details.