User:Toshio/Remove PyXML

PyXML has been dead upstream for quite a while. The current maintainer (rrakus) asked about removing it in February. The python stdlib has the main pieces of PyXML included as the xml module.

The Python stdlib in python2.x also has the dubious behaviour of importing PyXML if it is installed and replacing its own code with PyXMLs. In some cases, this leads to bugs as the old PyXML code does not cope with some usages that the version in the stdlib does.

Deps

From Fedora 17, current deps on PyXML

[david@f17-new ~]$ repoquery --tree-whatrequires PyXML
PyXML-0:0.8.4-27.fc17.x86_64 [cmd line]
 \_  SOAPpy-0:0.11.6-13.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 |   \_  python-twisted-web-0:11.1.0-2.fc17.x86_64 [1: SOAPpy = 0.11.6-13.fc17]
 |   |   \_  buildbot-master-0:0.8.6-2.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  buildbot-0:0.8.6-2.fc17.noarch [1: buildbot-master = 0.8.6-2.fc17]
 |   |   \_  cnucnu-0:0-0.9.20111030git315bba3b.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  deluge-common-0:1.3.5-1.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  deluge-0:1.3.5-1.fc17.noarch [1: deluge-common = 1.3.5-1.fc17]
 |   |   |   \_  deluge-console-0:1.3.5-1.fc17.noarch [1: deluge-common = 1.3.5-1.fc17]
 |   |   |   |   \_  deluge-0:1.3.5-1.fc17.noarch [1: deluge-console = 1.3.5-1.fc17]
 |   |   |   \_  deluge-daemon-0:1.3.5-1.fc17.noarch [1: deluge-common = 1.3.5-1.fc17]
 |   |   |   |   \_  deluge-0:1.3.5-1.fc17.noarch [1: deluge-daemon = 1.3.5-1.fc17]
 |   |   |   |   \_  deluge-console-0:1.3.5-1.fc17.noarch [1: deluge-daemon = 1.3.5-1.fc17]
 |   |   |   |   \_  deluge-gtk-0:1.3.5-1.fc17.noarch [1: deluge-daemon = 1.3.5-1.fc17]
 |   |   |   |   \_  deluge-web-0:1.3.5-1.fc17.noarch [1: deluge-daemon = 1.3.5-1.fc17]
 |   |   |   \_  deluge-gtk-0:1.3.5-1.fc17.noarch [1: deluge-common = 1.3.5-1.fc17]
 |   |   |   |   \_  deluge-0:1.3.5-1.fc17.noarch [1: deluge-gtk = 1.3.5-1.fc17]
 |   |   |   \_  deluge-web-0:1.3.5-1.fc17.noarch [1: deluge-common = 1.3.5-1.fc17]
 |   |   |   |   \_  deluge-0:1.3.5-1.fc17.noarch [1: deluge-web = 1.3.5-1.fc17]
 |   |   \_  flumotion-0:0.10.0-2.fc17.x86_64 [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  gphotoframe-0:1.5.1-1.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  gphotoframe-gss-0:1.5.1-1.fc17.noarch [1: gphotoframe = 1.5.1-1.fc17]
 |   |   \_  jabbim-0:0.5.1-4.svn20100612.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  ldtp-0:2.3.0-1.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  nordugrid-arc-arex-0:2.0.0-2.fc17.x86_64 [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  nordugrid-arc-aris-0:2.0.0-2.fc17.noarch [8: perl(Condor), perl(Fork), perl(LL), perl(LSF), perl(LogUtils), perl(PBS), perl(SGE), perl(SLURM)]
 |   |   |   |   \_  nordugrid-arc-arex-0:2.0.0-2.fc17.x86_64 [1: nordugrid-arc-aris = 2.0.0-2.fc17]
 |   |   |   |   \_  nordugrid-arc-egiis-0:2.0.0-2.fc17.x86_64 [1: nordugrid-arc-aris = 2.0.0-2.fc17]
 |   |   |   \_  pbstop-0:4.16-15.fc17.x86_64 [1: perl(PBS)]
 |   |   \_  nwsserver-0:2.0.0-4.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  openvswitch-test-0:1.4.0-5.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  openxcap-0:1.1.2-4.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  orbited-0:0.7.10-8.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  fedmsg-0:0.2.5-1.fc17.noarch [1: orbited = 0.7.10-8.fc17]
 |   |   |   |   \_  fedmsg-hub-0:0.2.5-1.fc17.noarch [1: fedmsg = 0.2.5-1.fc17]
 |   |   |   |   \_  fedmsg-irc-0:0.2.5-1.fc17.noarch [1: fedmsg = 0.2.5-1.fc17]
 |   |   |   |   \_  fedmsg-relay-0:0.2.5-1.fc17.noarch [1: fedmsg = 0.2.5-1.fc17]
 |   |   |   \_  moksha-0:0.8.8-2.fc17.noarch [1: orbited = 0.7.10-8.fc17]
 |   |   |   |   \_  fedmsg-0:0.2.5-1.fc17.noarch [1: moksha = 0.8.8-2.fc17]
 |   |   |   |   \_  moksha-doc-0:0.8.8-2.fc17.noarch [1: moksha = 0.8.8-2.fc17]
 |   |   |   |   \_  moksha-server-0:0.8.8-2.fc17.noarch [1: moksha = 0.8.8-2.fc17]
 |   |   |   \_  moksha-server-0:0.8.8-2.fc17.noarch [1: orbited = 0.7.10-8.fc17]
 |   |   \_  postr-0:0.12.4-6.fc17.x86_64 [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  pyicq-t-0:0.8.1.5-13.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  pyicq-t-mysql-0:0.8.1.5-13.fc17.noarch [1: pyicq-t = 0.8.1.5-13.fc17]
 |   |   \_  pyjigdo-0:0.4.0.3-4.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  python-Coherence-0:0.6.6.2-4.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  upnp-inspector-0:0.2.2-6.fc17.noarch [1: python-Coherence = 0.6.6.2-4.fc17]
 |   |   \_  python-foolscap-0:0.6.3-1.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  python-morbid-0:0.8.7.3-3.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  moksha-0:0.8.8-2.fc17.noarch [1: python-morbid = 0.8.7.3-3.fc17]
 |   |   |   \_  orbited-0:0.7.10-8.fc17.noarch [1: python-morbid = 0.8.7.3-3.fc17]
 |   |   \_  python-nevow-0:0.10.0-6.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  python-nova-0:2012.1-2.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  python-twisted-0:11.1.0-2.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  encuentro-0:0.5-7.fc17.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   \_  gadget-0:0.0.3-7.fc17.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   \_  moksha-0:0.5.0-5.fc15.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   \_  nwsserver-0:2.0.0-4.fc17.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   \_  pymodbus-0:0.9.0-2.fc17.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   \_  pympdtouchgui-0:0.327-3.fc17.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   \_  python-setuptools_trial-0:0.5.12-3.fc17.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   \_  python-txws-0:0.7-5.fc17.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   |   \_  moksha-0:0.8.8-2.fc17.noarch [1: python-txws = 0.7-5.fc17]
 |   |   |   \_  python-txzmq-0:0.5.0-2.fc17.noarch [1: python-twisted = 11.1.0-2.fc17]
 |   |   |   |   \_  moksha-0:0.8.8-2.fc17.noarch [1: python-txzmq = 0.5.0-2.fc17]
 |   |   \_  python-twisted-lore-0:11.1.0-2.fc17.x86_64 [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  python-twisted-0:11.1.0-2.fc17.noarch [1: python-twisted-lore = 11.1.0-2.fc17]
 |   |   \_  python-twisted-words-0:11.1.0-2.fc17.x86_64 [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  buildbot-master-0:0.8.6-2.fc17.noarch [1: python-twisted-words = 11.1.0-2.fc17]
 |   |   |   \_  jabbim-0:0.5.1-4.svn20100612.fc17.noarch [1: python-twisted-words = 11.1.0-2.fc17]
 |   |   |   \_  pyicq-t-0:0.8.1.5-13.fc17.noarch [1: python-twisted-words = 11.1.0-2.fc17]
 |   |   |   \_  python-twisted-0:11.1.0-2.fc17.noarch [1: python-twisted-words = 11.1.0-2.fc17]
 |   |   |   \_  python-wokkel-0:0.7.0-1.fc17.noarch [1: python-twisted-words = 11.1.0-2.fc17]
 |   |   \_  pywbem-0:0.7.0-6.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  cmpi-bindings-pywbem-0:0.4.17-2.fc17.x86_64 [1: pywbem = 0.7.0-6.fc17]
 |   |   |   \_  libstoragemgmt-0:0.0.9-1.fc17.i686 [1: pywbem = 0.7.0-6.fc17]
 |   |   |   |   \_  libstoragemgmt-devel-0:0.0.9-1.fc17.i686 [2: libstoragemgmt(x86-32) = 0.0.9-1.fc17, libstoragemgmt.so.0]
 |   |   |   \_  libstoragemgmt-0:0.0.9-1.fc17.x86_64 [1: pywbem = 0.7.0-6.fc17]
 |   |   |   |   \_  libstoragemgmt-devel-0:0.0.9-1.fc17.x86_64 [2: libstoragemgmt(x86-64) = 0.0.9-1.fc17, libstoragemgmt.so.0()(64bit)]
 |   |   |   \_  yawn-0:0-0.4.20120227svn561.fc17.noarch [1: pywbem = 0.7.0-6.fc17]
 |   |   \_  sslstrip-0:0.9-3.fc17.noarch [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   \_  syncevolution-1:1.2.2-1.fc17.x86_64 [1: python-twisted-web = 11.1.0-2.fc17]
 |   |   |   \_  syncevolution-devel-1:1.2.2-1.fc17.i686 [1: syncevolution = 1:1.2.2-1.fc17]
 |   |   |   \_  syncevolution-devel-1:1.2.2-1.fc17.x86_64 [1: syncevolution = 1:1.2.2-1.fc17]
 |   |   |   \_  syncevolution-gtk-1:1.2.2-1.fc17.x86_64 [1: syncevolution = 1:1.2.2-1.fc17]
 |   |   |   \_  syncevolution-meego-1:1.2.2-1.fc17.x86_64 [1: syncevolution = 1:1.2.2-1.fc17]
 |   |   |   \_  syncevolution-perl-1:1.2.2-1.fc17.x86_64 [1: syncevolution = 1:1.2.2-1.fc17]
 |   \_  pytrainer-0:1.7.2-3.fc17.noarch [1: SOAPpy = 0.11.6-13.fc17]
 \_  bkchem-0:0.14.0-4.pre2.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 \_  comoonics-base-py-0:0.1-6.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 |   \_  comoonics-cdsl-py-0:0.2-19.noarch [1: comoonics-base-py = 0.1-6.fc17]
 |   \_  comoonics-cluster-py-0:0.1-25.noarch [1: comoonics-base-py = 0.1-6.fc17]
 \_  comoonics-cdsl-py-0:0.2-19.noarch [1: PyXML = 0.8.4-27.fc17]
 \_  comoonics-cluster-py-0:0.1-25.noarch [1: PyXML = 0.8.4-27.fc17]
 |   \_  comoonics-cdsl-py-0:0.2-19.noarch [1: comoonics-cluster-py = 0.1-25]
 |   \_  osr-dracut-module-chroot-0:0.8-2.noarch [1: comoonics-cluster-py = 0.1-25]
 |   \_  osr-dracut-module-cluster-0:0.8-2.noarch [1: comoonics-cluster-py = 0.1-25]
 \_  eric-0:4.4.19-2.fc17.x86_64 [1: PyXML = 0.8.4-27.fc17]
 \_  fedora-business-cards-0:0.2.4.3-3.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 \_  grc-0:0.70-8.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 \_  heartbeat-0:3.0.4-1.fc17.2.x86_64 [1: PyXML = 0.8.4-27.fc17]
 |   \_  drbd-heartbeat-0:8.3.11-5.fc17.x86_64 [1: heartbeat = 3.0.4-1.fc17.2]
 |   \_  heartbeat-devel-0:3.0.4-1.fc17.2.i686 [1: heartbeat = 3.0.4-1.fc17.2]
 |   \_  heartbeat-devel-0:3.0.4-1.fc17.2.x86_64 [1: heartbeat = 3.0.4-1.fc17.2]
 |   \_  heartbeat-libs-0:3.0.4-1.fc17.2.i686 [1: heartbeat = 3.0.4-1.fc17.2]
 |   |   \_  heartbeat-devel-0:3.0.4-1.fc17.2.i686 [4: libapphb.so.2, libccmclient.so.1, libclm.so.1, libhbclient.so.1]
 |   \_  heartbeat-libs-0:3.0.4-1.fc17.2.x86_64 [1: heartbeat = 3.0.4-1.fc17.2]
 |   |   \_  heartbeat-0:3.0.4-1.fc17.2.x86_64 [2: libapphb.so.2()(64bit), libhbclient.so.1()(64bit)]
 |   |   \_  heartbeat-devel-0:3.0.4-1.fc17.2.x86_64 [4: libapphb.so.2()(64bit), libccmclient.so.1()(64bit), libclm.so.1()(64bit), libhbclient.so.1()(64bit)]
 \_  inksmoto-0:0.7.0-6.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 \_  libopensync-plugin-google-calendar-1:0.22-6.fc17.x86_64 [1: PyXML = 0.8.4-27.fc17]
 \_  openerp-0:6.1-1.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 \_  openxcap-0:1.1.2-4.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 \_  python-ZSI-0:2.0-12.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 |   \_  apbs-0:1.3-3.fc17.i686 [1: python-ZSI = 2.0-12.fc17]
 |   |   \_  apbs-devel-0:1.3-3.fc17.i686 [3: apbs = 1.3-3.fc17, libapbs.so.1, libapbsmainroutines.so.1]
 |   |   \_  apbs-devel-0:1.3-3.fc17.x86_64 [1: apbs = 1.3-3.fc17]
 |   |   \_  apbs-doc-0:1.3-3.fc17.x86_64 [1: apbs = 1.3-3.fc17]
 |   |   \_  apbs-tools-0:1.3-3.fc17.x86_64 [1: apbs = 1.3-3.fc17]
 |   |   \_  pymol-0:1.5.0.2-3.20120218svn3982.fc17.x86_64 [1: apbs = 1.3-3.fc17]
 |   |   |   \_  pymol-wxpython-0:1.5.0.2-3.20120218svn3982.fc17.x86_64 [1: pymol = 1.5.0.2-3.20120218svn3982.fc17]
 |   \_  apbs-0:1.3-3.fc17.x86_64 [1: python-ZSI = 2.0-12.fc17]
 |   |   \_  apbs-devel-0:1.3-3.fc17.i686 [1: apbs = 1.3-3.fc17]
 |   |   \_  apbs-devel-0:1.3-3.fc17.x86_64 [3: apbs = 1.3-3.fc17, libapbs.so.1()(64bit), libapbsmainroutines.so.1()(64bit)]
 |   |   \_  apbs-doc-0:1.3-3.fc17.x86_64 [1: apbs = 1.3-3.fc17]
 |   |   \_  apbs-tools-0:1.3-3.fc17.x86_64 [1: apbs = 1.3-3.fc17]
 |   |   \_  pymol-0:1.5.0.2-3.20120218svn3982.fc17.x86_64 [1: apbs = 1.3-3.fc17]
 |   \_  openerp-0:6.1-1.fc17.noarch [1: python-ZSI = 2.0-12.fc17]
 |   \_  sonata-0:1.6.2.1-6.fc17.x86_64 [1: python-ZSI = 2.0-12.fc17]
 \_  python-nova-0:2012.1.1-4.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 |   \_  openstack-nova-0:2012.1.1-4.fc17.noarch [1: python-nova = 2012.1.1-4.fc17]
 |   |   \_  openstack-nova-doc-0:2012.1.1-4.fc17.noarch [1: openstack-nova = 2012.1.1-4.fc17]
 |   |   \_  openstack-nova-novncproxy-0:0.3-11.fc17.noarch [1: openstack-nova = 2012.1.1-4.fc17]
 \_  salt-0:0.10.1-1.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 |   \_  salt-master-0:0.10.1-1.fc17.noarch [1: salt = 0.10.1-1.fc17]
 |   \_  salt-minion-0:0.10.1-1.fc17.noarch [1: salt = 0.10.1-1.fc17]
 \_  spacewalk-backend-tools-0:1.6.66-2.fc17.noarch [1: PyXML = 0.8.4-27.fc17]
 \_  subscription-manager-0:1.0.10-1.fc17.x86_64 [1: PyXML = 0.8.4-27.fc17]
 |   \_  subscription-manager-gui-0:1.0.10-1.fc17.x86_64 [1: subscription-manager = 1.0.10-1.fc17]
 |   |   \_  subscription-manager-firstboot-0:1.0.10-1.fc17.x86_64 [1: subscription-manager-gui = 1.0.10-1.fc17]
 |   \_  subscription-manager-migration-0:1.0.10-1.fc17.x86_64 [1: subscription-manager = 1.0.10-1.fc17]
 \_  synce-sync-engine-0:0.15.1-3.fc17.x86_64 [1: PyXML = 0.8.4-27.fc17]
 |   \_  libopensync-plugin-synce-wm5-0:0.15.1-3.fc17.x86_64 [1: synce-sync-engine = 0.15.1-3.fc17]
 \_  xen-0:4.1.2-20.fc17.x86_64 [1: PyXML = 0.8.4-27.fc17]
 |   \_  drbd-xen-0:8.3.11-5.fc17.x86_64 [1: xen = 4.1.2-20.fc17]
 |   \_  libvirt-daemon-xen-0:0.9.11.4-3.fc17.x86_64 [1: xen = 4.1.2-20.fc17]

Dep analysis

In Progress

Easy Fixes

• SOAPpy looks false. https://sourceforge.net/tracker/?func=detail&aid=3547666&group_id=26590&atid=387667
  • SOAPpy has a new upstream: http://pypi.python.org/pypi/SOAPpy/ https://github.com/kiorky/SOAPpy and PyXML is not listed as a dependency.
  • Plan to remove the dep in RawHide and see if anything breaks
  • Update in rawhide once F18 branches: https://bugzilla.redhat.com/show_bug.cgi?id=842786
• eric -- Looks like eric will fallback to a non-validating XML parser from the stdlib if PyXML is not installed. (Note that eric also requests a bugfix of PyXML which we should but aren't applying as a patch to our PyXML package https://bugzilla.redhat.com/show_bug.cgi?id=767737 ).
  • rdieter asks me to go ahead and try removing the dep in RawHide
• inksmoto looks false: Upstream seems to have ported from PyXML to python-lxml in the past. Code review supports that.
  • https://bugzilla.redhat.com/show_bug.cgi?id=842853
• python-nova looks false. Only uses minidom and parsers.expat which are in the stdlib. Nothing in the documentation mentions pyxml
  • https://bugzilla.redhat.com/show_bug.cgi?id=842866
• python-ZSI v2.0 uses PyXML. It looks like much (but not all) of the code has fallbacks to use locally maintained code if PyXML is not installed. python-ZSI-2.0a1 does not use PyXML. It looks like no one involved in PyXML is interested in making releases but we could package either the 2.0a1 or a snapshot to get rid of the PyXML dependency and pick up accumulated bugfixes.
• salt - Looks false. salt uses xml.dom.minidom which is in the python stdlib.
  • https://bugzilla.redhat.com/show_bug.cgi?id=842850
• xen looks like an old dep. Upstream has actively ported from PyXML to lxml in the past.
  • I found one instance where PyXML was still required for PrettyPrint().
  • Submitted bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=842843 with the following:
    • Remove PyXML Dep
    • Add python-lxml dep
    • Patch to use doc.toprettyxml() instead of pyxml's PrettyPrint(doc)

Require coding

• bkchem Looks like a single plugin currently needs PyXML (for xpath).
  • Reviewing the code, I find that it's using xml.minidom, xml.sax in many places. These are implemented in the python stdlib xml module. The bkchem progress.log talks about removing an unnecessary dep on pyxml so it seems to agree that for the main library, pyxml is optional.
  • bkchem/plugins/gtml.py ("GTML import-export plugin") has an import xml.xpath. This submodule is only in PyXML so we'll have to port it to some other xml library or not ship it. It looks fairly straightforward to port. In each case, it's using xpath.Evaluate to retrieve a list of nodes that match an expression and then iterating over them.

To be analyzed

• comoonics-base-py
• comoonics-cdsl-py
• comoonics-cluster-py
• grc
• libopensync-plugin-google-calendar
• openerp
• openxcap
• spacewalk-backend-tools
• subscription-manager
• synce-sync-engine

Fixed

• fedora-business-cards fixed
• heartbeat -- retired in F18