From Fedora Project Wiki

(Update)
(Updated)
Line 1: Line 1:
I am currently working to create some basic metrics to report on the activities on the [[BugZappers]]. My approach is to use the [https://fedorahosted.org/python-bugzilla/ python-bugzilla] package as much as possible.
I am currently working to create some basic metrics to report on the activities on the [[BugZappers]]. My approach is to use the [https://fedorahosted.org/python-bugzilla/ python-bugzilla] package as much as possible.
== Tools You Will Need==
* The [https://fedorahosted.org/python-bugzilla/ python-bugzilla] package.  (Try 'yum install python-bugzilla'
* [http://therabers.com/fedora/python-bugzilla/bzilla The modified bugzilla CLI] to add the --summary option.  I suggest saving this as ~/bin/bzilla (only because that is what I have done) 


== Proposed Metrics ==
== Proposed Metrics ==
Line 11: Line 15:
|# of Bugs 'Triaged' in the last xx days by Component || DONE || bzilla query --from-url='https://bugzilla.redhat.com/buglist.cgi?chfieldto=Now&query_format=advanced&chfield=keywords&chfieldfrom=-90d&chfieldvalue=Triaged&classification=Fedora&product=Fedora' --summary component
|# of Bugs 'Triaged' in the last xx days by Component || DONE || bzilla query --from-url='https://bugzilla.redhat.com/buglist.cgi?chfieldto=Now&query_format=advanced&chfield=keywords&chfieldfrom=-90d&chfieldvalue=Triaged&classification=Fedora&product=Fedora' --summary component
|-
|-
|# of Bugs 'Triaged' in the last xx days by Triager || In Process || Requires modification to bugzilla-python to enable gathering of 'history' data from bugzilla via XMLRPC. (The ability to get History data via XMLRPC was intrduced in BugZilla 3.4 and is marked as EXPERIMENTAL.)
|# of Bugs 'Triaged' in the last xx days by Triager || In Process || I am working on a patch to add the ability to query/display the bug history data via XMLRPC to the python-bugzilla package. This metric will provide the list of known active BugZappers.
|-
|-
|# of 'Triaged' bugs CLOSED in last xx Days || DONE || See: [https://bugzilla.redhat.com/buglist.cgi?keywords=Triaged&chfieldto=Now&query_format=advanced&chfield=bug_status&keywords_type=allwords&chfieldfrom=-90d&chfieldvalue=CLOSED&bug_status=CLOSED&classification=Fedora&product=Fedora]
|# of 'Triaged' bugs CLOSED in last xx Days || DONE || See: [https://bugzilla.redhat.com/buglist.cgi?keywords=Triaged&chfieldto=Now&query_format=advanced&chfield=bug_status&keywords_type=allwords&chfieldfrom=-90d&chfieldvalue=CLOSED&bug_status=CLOSED&classification=Fedora&product=Fedora]
Line 17: Line 21:
|# of 'Triaged' bugs CLOSED in last xx Days by Resolution || DONE || bzilla query --from-url='https://bugzilla.redhat.com/buglist.cgi?keywords=Triaged&chfieldto=Now&query_format=advanced&chfield=bug_status&keywords_type=allwords&chfieldfrom=-90d&chfieldvalue=CLOSED&bug_status=CLOSED&classification=Fedora&product=Fedora' --summary resolution
|# of 'Triaged' bugs CLOSED in last xx Days by Resolution || DONE || bzilla query --from-url='https://bugzilla.redhat.com/buglist.cgi?keywords=Triaged&chfieldto=Now&query_format=advanced&chfield=bug_status&keywords_type=allwords&chfieldfrom=-90d&chfieldvalue=CLOSED&bug_status=CLOSED&classification=Fedora&product=Fedora' --summary resolution
|-
|-
|# of 'Triaged' bugs CLOSED in last xx Days by # of days since Triaged flag was set || PENDING || To know if/how 'Triaging' is impacting the life expectancy of bugs. (This metric needs to be defined better)
|# of 'Triaged' bugs CLOSED in last xx Days by # of days since Triaged flag was set || PENDING || To know if/how 'Triaging' is impacting the life expectancy of bugs. (Need a better definition.  How about 'Average number of days from Triaged to CLOSED by resolution for 'Triaged' bugs CLOSED in last xx days')  
|-
|-
|List of 'Triaged' bugs currently marked as NEEDINFO || PENDING || This should generally be an empty list.  Bugs showing up here could be an indicator of  poor Triaging. See: [https://bugzilla.redhat.com/buglist.cgi?type0-1-0=allwords&field0-1-0=flagtypes.name&field0-0-0=flagtypes.name&classification=Fedora&query_format=advanced&value0-1-0=Triaged&bug_status=NEW&bug_status=ASSIGNED&bug_status=MODIFIED&bug_status=ON_DEV&type0-0-0=allwords&value0-0-0=NEEDINFO&product=Fedora]  (**NEED TO TEST THIS**)
|List of 'Triaged' bugs currently marked as NEEDINFO || PENDING || This should generally be an empty list.  Bugs showing up here could be an indicator of  poor Triaging. See: [https://bugzilla.redhat.com/buglist.cgi?type0-1-0=allwords&field0-1-0=flagtypes.name&field0-0-0=flagtypes.name&classification=Fedora&query_format=advanced&value0-1-0=Triaged&bug_status=NEW&bug_status=ASSIGNED&bug_status=MODIFIED&bug_status=ON_DEV&type0-0-0=allwords&value0-0-0=NEEDINFO&product=Fedora]  (Is this useful.  Remove?)
|-
|-
|# of bugs CLOSED by BugZappers in last xx days || PENDING || Need a list of BugZappers
|# of bugs CLOSED by BugZappers in last xx days || PENDING || Need a list of BugZappers
Line 28: Line 32:
|}
|}
== Additional Thoughts/Questions ==
== Additional Thoughts/Questions ==
* Who are the BugZappers?  Where is the list of members of the BugZappers? Proposed alternative is: BugZappers = BugZilla users that have made use of the 'Triaged' flag in the previous xx days.
* Q:Who are the BugZappers? A:BugZappers = Members of he Triagers group in FAS. But since this is not easy to scrape (and may included inactive persons) I am going to use the following definition. BugZappers = BugZilla users that have used the 'Triaged' flag in the previous 90 days.
* Eventually, these metrics should probably come from a SQL query.  Pulling all this data via XMLRPC will be slow.
* Eventually, these metrics should probably come from a SQL query.  Pulling all this data via XMLRPC could be slow.
* Pretty graphs could be created showing how these metrics change (or don't) over time.
* Pretty graphs could be created showing how these metrics change (or don't) over time.
* What other metrics should we be tracking?
* What other metrics should we be tracking?
* Should these metric be limited to bugs where Product=Fedora & Classification=Fedora?


== Current Metrics as of  June 22, 2010==
== Current Metrics as of  June 28, 2010==
Bugs Triaged in the last 60 Days: 327
  Bugs 'Triaged' in the last 60 Days: 229


   By Current Status:
   Summary of Bugs 'Triaged' in the last 60 days by Current Status:
       ASSIGNED - 60
       ASSIGNED - 54
       CLOSED - 65
       CLOSED - 40
       MODIFIED - 5
       MODIFIED - 1
       NEW - 184
       NEW - 132
       ON_QA - 11
       ON_QA - 2
      VERIFIED - 2


   By Component:
   Summary of Bugs 'Triaged' in the last 60 days by Component:
      NetworkManager - 1
       PackageKit - 19
       PackageKit - 19
       amarok - 1
       amarok - 2
       anaconda - 1
       anaconda - 1
      ant - 1
       banshee - 1
       banshee - 1
      cman - 1
       compiz - 2
       compiz - 5
       compiz-fusion - 1
       compiz-fusion - 1
      control-center - 1
      devhelp - 2
       digikam - 1
       digikam - 1
      distribution - 1
       epiphany - 1
       epiphany - 1
       evince - 2
       evince - 1
       evolution - 3
       evolution - 3
       fedora-bookmarks - 1
       fedora-bookmarks - 1
       firefox - 37
       firefox - 30
       gc - 1
       gcc - 1
      glib2 - 1
       glob2 - 1
       glob2 - 1
       gnome-desktop - 1
       gnome-desktop - 1
       gnome-panel - 1
       gnome-panel - 1
       gnome-session - 1
       gnome-session - 1
       initscripts - 5
       initscripts - 4
      iptables - 1
       java-1.6.0-openjdk - 1
       java-1.6.0-openjdk - 1
       k3b - 1
       k3b - 1
      kcm-gtk - 1
       kde-plasma-networkmanagement - 2
       kde-plasma-networkmanagement - 2
       kdeaccessibility - 1
       kdeaccessibility - 1
       kdebase - 1
       kdebase - 1
       kdebase-workspace - 3
       kdebase-workspace - 1
       kdegraphics - 1
       kdegraphics - 1
      kdelibs - 1
       kdenetwork - 1
       kdenetwork - 1
       kdepim - 3
       kdepim - 3
       kernel - 6
      kdeutils - 1
       kernel - 2
       knetworkmanager - 2
       knetworkmanager - 2
       libX11 - 4
       libX11 - 2
       libXfixes - 1
       libXfixes - 1
       libdrm - 1
       libdrm - 1
      libtiff - 2
       livecd-tools - 1
       livecd-tools - 1
       mesa - 5
       me-tv - 1
      nautilus - 3
       mesa - 2
       nspluginwrapper - 1
       phonon - 1
       phonon - 1
       pixman - 1
       pixman - 1
       preupgrade - 1
       preupgrade - 1
      quodlibet - 1
       seamonkey - 4
      releng - 1
       strigi - 6
      rgmanager - 1
       seamonkey - 6
      spectrum - 1
       strigi - 3
       sugar - 1
       sugar - 1
       system-config-display - 2
       system-config-display - 1
       system-config-users - 1
       system-config-users - 1
      system-setup-keyboard - 1
       thunderbird - 5
       thunderbird - 14
       xine-lib - 2
       xine-lib - 2
       xine-ui - 1
       xine-ui - 1
      xorg-x11-apps - 1
       xorg-x11-drv-ati - 26
       xorg-x11-drv-ati - 29
       xorg-x11-drv-evdev - 1
       xorg-x11-drv-evdev - 2
       xorg-x11-drv-intel - 22
       xorg-x11-drv-intel - 42
       xorg-x11-drv-nouveau - 24
       xorg-x11-drv-nouveau - 32
      xorg-x11-drv-nv - 1
       xorg-x11-drv-radeonhd - 1
       xorg-x11-drv-radeonhd - 1
       xorg-x11-drv-sis - 1
       xorg-x11-drv-sis - 1
       xorg-x11-drv-synaptics - 1
       xorg-x11-drv-synaptics - 1
      xorg-x11-drv-v4l - 1
      xorg-x11-drv-vesa - 2
       xorg-x11-drv-vmmouse - 3
       xorg-x11-drv-vmmouse - 3
      xorg-x11-drv-vmware - 1
       xorg-x11-drv-wacom - 6
       xorg-x11-drv-wacom - 6
      xorg-x11-fonts - 2
       xorg-x11-server - 12
       xorg-x11-server - 30
       xulrunner - 1
       xulrunner - 3
      yelp - 1
       yum - 1
       yum - 1
      zlib - 4
  # of 'Triaged' bugs CLOSED in last 60 Days: 476
  Summary of 'Triaged' bugs CLOSED in last 60 Days by Resolution:
      CANTFIX - 4
      CURRENTRELEASE - 21
      DEFERRED - 1
      DUPLICATE - 23
      ERRATA - 71
      INSUFFICIENT_DATA - 53
      NEXTRELEASE - 15
      NOTABUG - 13
      RAWHIDE - 147
      UPSTREAM - 5
      WONTFIX - 101
      WORKSFORME - 22

Revision as of 05:16, 29 June 2010

I am currently working to create some basic metrics to report on the activities on the BugZappers. My approach is to use the python-bugzilla package as much as possible.

Tools You Will Need

Proposed Metrics

Description Status Comments/Notes/Reference
# of Bugs 'Triaged' in the last xx days DONE See: [1]
# of Bugs 'Triaged' in the last xx days by Current Status DONE bzilla query --from-url='https://bugzilla.redhat.com/buglist.cgi?chfieldto=Now&query_format=advanced&chfield=keywords&chfieldfrom=-90d&chfieldvalue=Triaged&classification=Fedora&product=Fedora' --summary bug_status
# of Bugs 'Triaged' in the last xx days by Component DONE bzilla query --from-url='https://bugzilla.redhat.com/buglist.cgi?chfieldto=Now&query_format=advanced&chfield=keywords&chfieldfrom=-90d&chfieldvalue=Triaged&classification=Fedora&product=Fedora' --summary component
# of Bugs 'Triaged' in the last xx days by Triager In Process I am working on a patch to add the ability to query/display the bug history data via XMLRPC to the python-bugzilla package. This metric will provide the list of known active BugZappers.
# of 'Triaged' bugs CLOSED in last xx Days DONE See: [2]
# of 'Triaged' bugs CLOSED in last xx Days by Resolution DONE bzilla query --from-url='https://bugzilla.redhat.com/buglist.cgi?keywords=Triaged&chfieldto=Now&query_format=advanced&chfield=bug_status&keywords_type=allwords&chfieldfrom=-90d&chfieldvalue=CLOSED&bug_status=CLOSED&classification=Fedora&product=Fedora' --summary resolution
# of 'Triaged' bugs CLOSED in last xx Days by # of days since Triaged flag was set PENDING To know if/how 'Triaging' is impacting the life expectancy of bugs. (Need a better definition. How about 'Average number of days from Triaged to CLOSED by resolution for 'Triaged' bugs CLOSED in last xx days')
List of 'Triaged' bugs currently marked as NEEDINFO PENDING This should generally be an empty list. Bugs showing up here could be an indicator of poor Triaging. See: [3] (Is this useful. Remove?)
# of bugs CLOSED by BugZappers in last xx days PENDING Need a list of BugZappers
# of bugs CLOSED by BugZappers in last xx days by BugZapper PENDING Need a list of BugZappers
# of bugs CLOSED by BugZappers in last xx days by Resolution PENDING Need a list of BugZappers

Additional Thoughts/Questions

  • Q:Who are the BugZappers? A:BugZappers = Members of he Triagers group in FAS. But since this is not easy to scrape (and may included inactive persons) I am going to use the following definition. BugZappers = BugZilla users that have used the 'Triaged' flag in the previous 90 days.
  • Eventually, these metrics should probably come from a SQL query. Pulling all this data via XMLRPC could be slow.
  • Pretty graphs could be created showing how these metrics change (or don't) over time.
  • What other metrics should we be tracking?
  • Should these metric be limited to bugs where Product=Fedora & Classification=Fedora?

Current Metrics as of June 28, 2010

  Bugs 'Triaged' in the last 60 Days: 229
  Summary of Bugs 'Triaged' in the last 60 days by Current Status:
     ASSIGNED - 54
     CLOSED - 40
     MODIFIED - 1
     NEW - 132
     ON_QA - 2
  Summary of Bugs 'Triaged' in the last 60 days by Component:
     PackageKit - 19
     amarok - 2
     anaconda - 1
     banshee - 1
     compiz - 2
     compiz-fusion - 1
     digikam - 1
     epiphany - 1
     evince - 1
     evolution - 3
     fedora-bookmarks - 1
     firefox - 30
     gcc - 1
     glib2 - 1
     glob2 - 1
     gnome-desktop - 1
     gnome-panel - 1
     gnome-session - 1
     initscripts - 4
     java-1.6.0-openjdk - 1
     k3b - 1
     kcm-gtk - 1
     kde-plasma-networkmanagement - 2
     kdeaccessibility - 1
     kdebase - 1
     kdebase-workspace - 1
     kdegraphics - 1
     kdenetwork - 1
     kdepim - 3
     kdeutils - 1
     kernel - 2
     knetworkmanager - 2
     libX11 - 2
     libXfixes - 1
     libdrm - 1
     libtiff - 2
     livecd-tools - 1
     me-tv - 1
     mesa - 2
     phonon - 1
     pixman - 1
     preupgrade - 1
     seamonkey - 4
     strigi - 6
     sugar - 1
     system-config-display - 1
     system-config-users - 1
     thunderbird - 5
     xine-lib - 2
     xine-ui - 1
     xorg-x11-drv-ati - 26
     xorg-x11-drv-evdev - 1
     xorg-x11-drv-intel - 22
     xorg-x11-drv-nouveau - 24
     xorg-x11-drv-radeonhd - 1
     xorg-x11-drv-sis - 1
     xorg-x11-drv-synaptics - 1
     xorg-x11-drv-vmmouse - 3
     xorg-x11-drv-vmware - 1
     xorg-x11-drv-wacom - 6
     xorg-x11-server - 12
     xulrunner - 1
     yum - 1
     zlib - 4
  # of 'Triaged' bugs CLOSED in last 60 Days: 476
  Summary of 'Triaged' bugs CLOSED in last 60 Days by Resolution:
     CANTFIX - 4
     CURRENTRELEASE - 21
     DEFERRED - 1
     DUPLICATE - 23
     ERRATA - 71
     INSUFFICIENT_DATA - 53
     NEXTRELEASE - 15
     NOTABUG - 13
     RAWHIDE - 147
     UPSTREAM - 5
     WONTFIX - 101
     WORKSFORME - 22