Desktop Team Triage Policy

Informational material for purposes of a discussion on fedora-advisory-list—not finished and under review!

The purpose of this document is to stabilize the current policy of handling bugs on the triaging side of the pipeline (i.e., I won't deal here with MODIFIED, ON_QA and similar states of bugs; if needed, this document certainly can be extended for dealing with whole pipeline), so basically at the moment a bug is marked as ASSIGNED, it falls out of the scope of this document.

The structure of this document follows a bug from its entering our bugzilla to either the moment the bug is ASSIGNED to particular developer or until it is CLOSED for some reason other than that it is resolved.

NEW

Reporter files a new bug against some particular component and the bug gets automatically assigned to the default owner of the component.

Practically a bug in this state means that the bug hasn't been triaged and that the people who are responsible for triaging bugs should deal with it. In the ideal state (i.e., there are enough triagers available), developers shouldn't touch such bugs at all. Of course, it doesn't mean that developers are not allowed to assign a bug to themselves, thus taking it out of the dirty hands of bug triagers -- the whole point of bug triage is to make bugs digestable by developers, and when a developer decides that he knows enough about a bug and wants to take it himself, there is no reason in the world to stop him from doing so. The only thing to be emphasized is that when developer takes over a bug, he should switch status of the bug to ASSIGNED and then he is on his own.

Because bug 234261 has been fixed (yay!) we can finally say that the bug is NEW as long as it has been not triaged and only then (manually) will be switched to ASSIGNED.

[unrelated discussion about internal organization of bugs and assigning them to different developers was snipped out here]

Possible transitions from NEW state

• ASSIGNED -- the bug has been triaged, the way to reproduce it was found and described, and comments/attachments to the bug contain all information which the reporter is able to produce. For hardware related bugs (e.g., most xorg bugs unfortunately), the requirement of reproduction cannot be usually satisfied (unless by chance a triager has the same hardware as the reporter).

  Generally speaking, ASSIGNED bug means that bug triager won't touch such bugs anymore with the exception of old NEEDINFO bugs (see below).

• NEEDINFO -- no! This should not be considered a valid transition from any state, IMHO, but always a temporary state after which the bug returns to its current base state. See below for more discussion about NEEDINFO and standardized comments for setting it on.
• CLOSED/INSUFFICIENT_DATA -- despite what the training materials for Bugzilla say, this is not automatic (thankfully) and it has to be set manually for each bug (or using mass-changes in bugzilla queries). The two main reasons why this is used are when the reporter loses interest in the bug (e.g., because of our delays in replying, which should happen less and less) and never replies to the questions, or when they lose access to the important piece of hardware for reproduction of the bug.

  Currently the process is that the special query is set which shows all bugs in NEEDINFO state for a month, and then bugmaster goes through this list daily (or at least every other day) and manually decides what to do with these bugs (one more notice, close as INSUFFICIENT_DATA, switch to ASSIGNED because all information was actually provided and only reporter forgot to check proper checkbox).

  When the bug should be closed with this resolution, a standard comment is added:

  Since there are insufficient details provided in this report for us to investigate the issue further, and we have not received feedback to the information we have requested above, we will assume the problem was not reproducible, or has been fixed in one of the updates we have released for the reporter's distribution.

  Users who have experienced this problem are encouraged to upgrade to the latest update of their distribution, and if this issue turns out to still be reproducible in the latest update, please reopen this bug with additional information.

  Closing as INSUFFICIENT_DATA.

• CLOSED/CANTFIX -- most often used for binary-only drivers in Xorg, where this comment is added (for ATI drivers, which is less frequent, I add only the first paragraph):

  Thanks for the report. We are sorry that we cannot help you with your problem, but we are not able to support binary-only drivers. If you would be able to reproduce this issue using only open source software, please, reopen this bug with the additional information, but in meantime I have no choice than to close this bug as CANTFIX (because we really cannot fix it).

  For users who are experiencing problems installing, configuring, or using the unsupported 3rd party proprietary nvidia video driver, Nvidia provides indirect customer support via an online web based support forum. Nvidia monitors these web forums for commonly reported problems and passes them on to Nvidia engineers for investigation. Once they've isolated a particular problem, it is often fixed in a future video driver update.

  The NVNews Nvidia Linux driver forum is located at:

  http://www.nvnews.net/vbulletin/forumdisplay.php?s=&forumid=14

  Once you have reported this issue in the Nvidia web forums, others who may have experienced the particular problem may be able to assist. If there is a real bug occuring, Nvidia will be able to determine this, and will likely resolve the issue in a future driver update for the operating system releases that they officially support.

  While Red Hat does not support the proprietary nvidia driver, users requiring technical support may also find the various X.Org, XFree86, and Red Hat mailing lists helpful in finding assistance:

  X.Org mailing lists:

  http://www.freedesktop.org/XOrg/XorgMailingLists

  XFree86 mailing lists:

  http://www.xfree86.org/sos/lists.html

  Red Hat mailing lists:

  https://listman.redhat.com/mailman/listinfo

• CLOSED/CURRENTRELEASE (or CLOSED/RAWHIDE) -- used for Fedora bugs (RHEL bugs have to go through QA etc.) when the bug is fixed in the updated release of the package or in Rawhide. Also a reaction to the bug when the reporter, after getting NEEDINFO with a request for reproduction on the updated system, is not able to reproduce the bug. There is not much standard about the closing comment, if any comment is needed at all (which is not the case in most situations), usually something in the style of this is enough (this is more for the reporter's inability to reproduce a bug from older distribution):

  This issue has been already resolved in the latest version of the package (NAME-OF-THE-PACKAGE). Please, reopen if you are able to reproduce this with currently supported distribution.

• CLOSED/NOTABUG -- some use of this resolution is obvious: the reported issue is not a bug at all, just reporter made a mistake in using the program, or there are some other factors which caused the problem (faulty hardware, configuration, etc.).

  Other uses of this resolution is more tricky. The issue is when the triager despite all best efforts is unable to reproduce the bug. Either the reason for such inability is obvious and appropriate resolution could be used (e.g., CLOSED/CURRENTRELEASE), but quite often it isn't. Then this resolution is appropriate. (or should it be CLOSED/WORKSFORME?)

Dark magic of NEEDINFOing

Although I claimed that NEEDINFO is not valid transition state, it doesn't mean by any means that NEEDINFO is not important, or that it shouldn't be used. Just to the contrary, for the bug triager it is the most important state that the bug under her control could be in.

The main reason why I don't want to talk about NEEDINFO as another transition state is to emphasize that the bug in NEEDINFO state doesn't leave bug triagers' control -- it should be kept under control and if it is not alive enough, it should be closed (more about that later).

Putting a bug into NEEDINFO will get it into something I would call The Iron Grip of NEEDINFO. The point is that one of the bug triager's regular tasks is to run daily a query old NEEDINFO which finds all bugs in the NEEDINFO status in (currently) Xorg, Gecko, and Evolution super-components which were not changed in the last 30 days. Then his duty is to do something about each of this bug:

• close as CLOSED/INSUFFICIENT_DATA,
• ASSIGN to developer (e.g., it happens that NEEDINFO state was not cleared even though the information was provided), or if neither of these is appropriate,
• do something else,
• make a comment, any comment, at least the one threating with closing.

The point is obviously that something MUST change in the bug, so that the bug won't show up next day in the same query. Of course, if the reporter gives reasonable explanation for not being able to answer immediately, the bug triager should be free to allow him to do that, but always in one month installments with pinging him at least every month. By keeping one month as rigorous schedule for looking at every NEEDINFO bug we can be sure that the bug doesn't slip out of control.

The question is how many times a bug can show up in the query (i.e., how many months it can get saved from being closed). And the answer is very vague. Usually, the bug gets at least once a comment which explitly says, that if the answer isn't provided within a month, it will be closed as CLOSED/INSUFFICIENT_DATA, but judgement when the bowl breaks is not rigid. If there are clear signs that the bug is useless and the answer won't come or it will be useless (e.g., a year old bug without a comment in the last ten months, in a component which was drastically changed in the last year), then probably even no reminder need to be sent. However, bugs in RHEL-like distribution with grave impact may get multiple reminders, direct email/IRC can be tried, before the triager's patience will run out.

This iron grip of NEEDINFO is so useful in eliminating incovenient bugs, that the basic strategy of dealing with any problematic bug for any reason, is to convert it into NEEDINFO state, and then it will be dealt with automatically. Flip side of this is that every developer should be aware that a bug in NEEDINFO state is on the slippery slope towards being closed in the limited amount of time. (Of course, developers decisions are sacred, so if needed comment Please, don't close ever. could be added to avoid closing.)

[Discussion of RHEL and RHEL-like bugs omitted.]

One important aspect of NEEDINFO is that it is easy to create standardized responses. There are many standard comments for switching a bug to NEEDINFO state. This is the list:

• Xorg asking for more information:

  Thanks for the bug report. We have reviewed the information you have provided above, and there is some additional information we require that will be helpful in our diagnosis of this issue.

  Please attach your X server config file (/etc/X11/xorg.conf) and X server log file (/var/log/Xorg.*.log) to the bug report as individual uncompressed file attachments using the bugzilla file attachment link below.

  Could you please also try to run without any /etc/X11/xorg.conf whatsoever and let X11 autodetect your display and video card? Attach to this bug /var/log/Xorg.0.log from this attempt as well, please.

  We will review this issue again once you've had a chance to attach this information.

  Thanks in advance.

• Asking for crash report from firefox:

  Thanks for the bug report. We have reviewed the information you have provided above, and there is some additional information we require that will be helpful in our diagnosis of this issue.

  Please install firefox-debuginfo; in order to do this you have to enable -debuginfo repository.

  yum install --enablerepo=\*debuginfo firefox-debuginfo

  (if you use x86_64 firefox, install firefox-debuginfo.x86_64 package).

  Then run firefox with a parameter -g. That will start firefox running inside of gdb debugger. Then use command run and do whatever you did to make firefox crash. When it happens, you should go back to the gdb and run

  (gdb) thread apply all backtrace

  This produces usually many screens of the text. Copy all of them into a text editor and attach the file to the bug as an uncompressed attachment.

  We will review this issue again once you've had a chance to attach this information.

  Thanks in advance.

• Trying to force activity from reporter (when the situation is not ready for CLOSED/INSUFFICIENT_DATA yet):

  Reporter, could you please reply to the previous question? If you won't reply in one month, I will have to close this bug as INSUFFICIENT_DATA. Thank you.

• There is also a message asking reporter to attempt to reproduce bug after installing all current updates of her distribution (I have also developed a tendency to add to all NEEDINFOs a threat about possible closing after a predetrmined period of time, especially when I suspect high possibility of non-answer):

  Since this bugzilla report was filed, there have been several major updates, which may have resolved this issue. Users who have experienced this problem are encouraged to upgrade their system to the latest version of their distribution available.

  Please, if you experience this problem on the up-to-date system, let us now in the comment for this bug, or whether the upgraded system works for you.

  If you won't be able to reply in one month, I will have to close this bug as INSUFFICIENT_DATA. Thank you.

• Special suicide NEEDINFO for asking reporter to upstream bug (of course, http://bugzilla.mozilla.org needs to be replaced by particular bug database -- http://bugzilla.gnome.org or http://bugzilla.freedesktop.org):

  If this issue turns out to still be reproduceable in the latest updates for this Fedora Core release, please file a bug report in the the upstream bugzilla located at http://bugzilla.mozilla.org in the particular component.

  Once you've filed your bug report to the upstream bugzilla, if you paste the new bug URL here, Red Hat will continue to track the issue in the centralized upstream bug tracker, and will review any bug fixes that become available for consideration in future updates.

  Setting status to NEEDINFO, and awaiting upstream bug report URL for tracking.

  Thanks in advance.

  When this NEEDINFO request is fulfilled, then the number of the upstream bug has to be inserted in External Bugzilla References section of the bug page, and the bug is CLOSED/UPSTREAM with this comment:

  We believe it is more appropriate to let this bug be resolved upstream. Red Hat will continue to track the issue in the centralized upstream bug tracker, and will review any bug fixes that become available for consideration in future updates.

  Thank you for the bug report.

Bugs retention policy

(currently this policy so far applies only to Xorg, Evolution, and Gecko & co. bugs)

Although we may like to keep bugs open for as long as they are solved, Red Hat operates only with limited resources (which is especially true about the desktop team), and so we have to limit the number of bugs which are currently open (and where we implicitly promise that some development will be done). Therefore old bugs have to be removed out of the way.

• Fedora-test bugs have to managed -- when the distribution being tested is released, all bugs against *-test[1234] product should be either closed or moved to the released distribution itself. For example, when F10 is released, F10-test* bugs should be either closed or moved to F10. When the next distribution is released (e.g., F11 in our case) all these test bugs will be CLOSED with please, reproduce and reopen if still applicable:

  Fedora Core 5 and Fedora Core 6 are, as we're sure you've noticed, no longer test releases. We're cleaning up the bug database and making sure important bug reports filed against these test releases don't get lost. It would be helpful if you could test this issue with a released version of Fedora or with the latest development / test release. Thanks for your help and for your patience.

  [This is a bulk message for all open FC5/FC6 test release bugs. I'm adding myself to the CC list for each bug, so I'll see any comments you make after this and do my best to make sure every issue gets proper attention.]

  Therefore, all bugs which are just placeholders for future action, notes about things to investigate etc., should never be filed against *test* version, and actually it probably shouldn't be filed against any other version than against Rawhide.

• The same applies to obsolete bugs. After Fedora Legacy shut down, the policy is that a distribution is declared as obsolete one month after release of the n+2 distribution (e.g., one month after release of Fedora 7 for Fedora Core 5 bugs). When this happens, the following comment (with changed version numbers, of course) is added to all applicable bugs:

  Fedora Core 5 is no longer supported, could you please reproduce this with the updated version of the currently supported distribution (Fedora Core 6, or Fedora 7, or Rawhide)? If this issue turns out to still be reproducible, please let us know in this bug report. If after a month's time we have not heard back from you, we will have to close this bug as CANTFIX.

  Setting status to NEEDINFO, and awaiting information from the reporter.

  Thanks in advance.

  After a month of waiting (so, actually reporters get two months for liquidation of obsolete bugs), all such bugs which are still opened are closed with a message like this:

  We haven't got any reply to the last question about reproducability of the bug with Fedora Core 6, Fedora 7, or Fedora devel. Mass closing this bug, so if you have new information that would help us fix this bug, please reopen it with the additional information.