QA Improve reporting
These are general notes that I have compiled on how we could significantly improve reporting.
Feel free to comment on this and add what ever you feel is lacking from it etc..
Bugzilla reporting UI to complicated.
When a new reporters enters Bugzilla for the first time the current reporting interface might seem to overwhelming to the reporter which leads to all kinds of trouble.
Resolution: Remove/hide or lock ( fields can be visible set to a certain value but none editable ) all unnecessary fields from the reporter in the Bugzilla UI if a reporter does not belong to any group. ( if a reporter is in x group it might show and allow him to edited certain fields based on what that x group is )
QA Task: Identify which field are unnecessary.
Note since many of us have extended privileges in bugzilla an test user without any privileges needs to exist so we are able to correctly identify which field(s) need locking or hiding.
List of minimal forms that need to be present and editable by the novice reporter.
Component
Component Description
Platform # If present selectable values should only be All,i386 and x86_64
Version # Selectable value should be restricted to supported releases + Rawhide
URl
Summary
Description
Attachment
Security sensitive bug
External Bug
QA Task: After consensus has been reached on which fields can be removed/hide or locked work with Bugzilla maintainers to see this get implemented.
Note this is being handled upstream
See https://bugzilla.redhat.com/show_bug.cgi?id=506936
Upstream bug
https://bugzilla.mozilla.org/show_bug.cgi?id=376673
Blog post from maintainer with the new simple interface..
http://lpsolit.wordpress.com/2009/05/24/simpler-form-to-file-bugs-in-bugzilla-3-4/
Lack of needed information for maintainer(s) to be able to successfully work with the report.
The reason for the lack of the needed information on a report is in most cases simply that the reporter has not a clue what to include in the report itself and is not mandatory to provide that information.
The underlying problem is not the reporter nor the triager which often ask the reporter to include wrong information because the triager has no better clue what to include in the report so he ask for the most common include case ( usually to include /var/log/messages). The problem is that the maintainer(s) them self have failed to provide this information or has done so only to the reporter on a report bases.
When a maintainer introduces a component into Fedora it should be mandatory for him to provide this information along with how to enable debug output and to provide test plans for the component.
Including the test plans however are not relevant to this proposal.
QA Task: Contact FESCO and ask them to set this rule and enforce it.
( New components will not be let into Fedora without this information )
Which leaves us with how do we handle existing components in Fedora?
To handle the existing components in Fedora I propose that we file a report to all components in bugzilla requesting that maintainer(s) provides us with that information.
If they do not respond to that report that we file the component in question will enter the removal phase.
You might be wondering why I propose to use Bugzilla vs other alternatives like contacting the maintainer directly.
Simply to for rough measurements and to keep track of the response.
QA Task: Come up with msg and file a report against components in Bugzilla.
The maintainers have provided us with that information then what?
After we have received the needed information from maintainer we insert that information into a DB later to be extracted by bugzilla present it to the reporter and be made mandatory for the reporter to include the files maintainer has requested to have in his report.
QA Task: Come up with how we like to store that information in a DB
QA Task: Contact infrastructure for resources
QA Task: Work with Bugzilla maintainers on the best way to present this information to the reporter
QA Task: Find out how ABRT does this..
Duplicated reports
The reason for reporters is they simply are to lazy or don't possess the know how on how to use the search utility in Bugzilla.
To reduce the risk of duplicated reports we need to provide a predefined search of the component in Bugzilla or an url ( https://admin.fedoraproject.org/pkgdb/packages/bugs/$component ) which the reporter could just click on or even better would be if we could show the reporter latest x number of entry's on the report page against the component he's about to report.
QA Task: Work with Bugzilla maintainers for the best solution.
Filed an upstream bug [RFE] Add an "Component Recent Reports" element to the reporters UI
https://bugzilla.mozilla.org/show_bug.cgi?id=499320
Suggested new mockup on the new simplified report form
https://bug499320.bugzilla.mozilla.org/attachment.cgi?id=384113
Unresponsive reporter
When a reporter has not replied to a question and the maintainers does not want to close the bug he can file a request ticket in fedora-qa or send a mail to the test-list requesting for assistance.
QA Task: Better advertise which service/resources are for maintainers disposal from QA.
Python-Bugzilla
Afaik example code is missing for maintainers to be able to integrate this with their application.
QA Task: Provide maintainers with example code to use
Lower reporters expectations
Novice reporters tend to have too high expectations after reporting and are expecting immediate treatment from maintainer(s) When they do not get what they expect they tend to get frustrated usually start messing with severity and priority field to get some response from maintainer and when they dont get it they never file a report again.
One pretty simple way to address this issue is when a report is filed and automated stock response is sent to the reporter that explains to the reporter that his report might not be work on right away due to load on maintainer(s).
Remove the severity/priority field from reporter.
QA Task: Work with bugzilla maintainers in removing the fields and set the stock response.