From Fedora Project Wiki
(Redirect Package Maintainer wiki links to docs.fp.o)
 
(23 intermediate revisions by 4 users not shown)
Line 8: Line 8:
'''Solution''': instead of manual change (also provides might be wrong) can be version normalized by Perl itself (module version can do it).
'''Solution''': instead of manual change (also provides might be wrong) can be version normalized by Perl itself (module version can do it).


* After taking over Perl dependency generator
* After taking over Perl dependency generator ✓
* Macros for simplified packaging
* Macros for simplified packaging
** the version translator
** the version translator
*** Perl architecture independent (because of upgrading perl): perl-Fedora-VSP ✓
*** C implementation for build-requires required by perl-srpm-macros
** the ''perl()'' wrapper
** the ''perl()'' wrapper
** common snippets for EU::MM, Module::Build, etc.
** common snippets for EU::MM, Module::Build, etc.
* <code>version->parse->normal</code> normalization in macros and generator
* <code>version->parse->normal</code> normalization in macros and generator
* Normalize RPM package version?
* Normalize RPM package version?
** Teach [[Upstream_release_monitoring | Fedora upstream monitoring]] ''cnucnu(1)''
** Teach [https://docs.fedoraproject.org/en-US/package-maintainers/Upstream_Release_Monitoring/ Fedora upstream monitoring] ''cnucnu(1)''
 
== Macros for simplified packaging ==
'''Problem''': There is still lot of manual work in specfiles. Better macros could improve packaging. Some packagers might discourage changes, because they prefer lot of ifdefs for different EL releases.
 
'''Solution''': Improvements above could make maintenance easier for future. One macro in macro file can redefine content of all specfiles and rebuild would add all changes.
 
* Version normalizer (see first paragraph)
* perl() wrapper
** This will mask SCL modifications
* Dependency filter abstraction
** One syntax for Fedora and EPEL
** One syntax for SCL
* Provide standard EU::MM, Module::Build, Module::Bild::Tiny, &c. %build and %install implementations
* Standard implementations for CPAN packages
** ''URL''
** ''Sourece0''
** ''%prep''
* Pack files scriptlets editor
** Allow Fatpacker and inc::Module::Build bundler to work on packaged modules
* Mark POD files as documentation


== Dependency generator ==
== Dependency generator ==
Problem: dependency generator in RPM has lot of incorrect data, which leads to complicated filtering of provides/requires
'''Problem''': dependency generator in RPM has lot of incorrect data, which leads to complicated filtering of provides/requires


Solution: dependency generator will be removed from RPM and owned by Perl team.  
'''Solution''': dependency generator will be removed from RPM and owned by Perl team.  


Comment: More improvements in main generator are not possible because contyk's smarter scaner has many dependencies. At least contyk's scaner could be used for generating new packages and updates.
'''Comment''': More improvements in main generator are not possible because contyk's smarter scaner has many dependencies. At least contyk's scaner could be used for generating new packages and updates.


* Take over dependency generator. Assignee: [[User:jplesnik]], [[User:mmaslano]]
* Take over dependency generator. Assignee: [[User:jplesnik]], [[User:mmaslano]]. Packaged as ''perl-generators''. ✓
* Tests. Assignee: [[User:jplesnik]]
* Tests. Assignee: [[User:jplesnik]]
* Support no, base, parent, Moose's with, POE's extensions, Test::Requires, other frameworks, see Module::ScanDeps::scan_chunk()
* Support no, base, parent, Moose's with, POE's extensions, Test::Requires, other frameworks, see Module::ScanDeps::scan_chunk()
Line 32: Line 54:


== Fedora-Rebuild ==
== Fedora-Rebuild ==
'''Problem''': annual Perl rebuild took more than month. No tooling is provided by Fedora nor rpm/yum-tools.
'''Solution''': Fedora::Rebuild is able to count build order of packages based on their dependencies.
'''Comment''': rebuilds of every new release should make easier finding bugs in other packages sooner and make rebuild shorter.
* Assignee: [[User:ppisar]]
* Assignee: [[User:ppisar]]
* Optimize pure-perl resolver
* Optimize pure-perl resolver
** Finished in git tree. ✓
** Finished in git tree. ✓
** Release new version.
** Release new version.
* Implement [[Features/Hawkey | hawkey]]-based resolver
* Implement [[Features/Hawkey | hawkey]]-based resolver
** XS wrapper for hawkey
** XS wrapper for hawkey
Line 45: Line 73:
** Select binary packages from Copr's YUM repository
** Select binary packages from Copr's YUM repository
** Requires YUM downloaded and metadata parser
** Requires YUM downloaded and metadata parser
* Mock initialization with multiple groups and packages (to support rebuilding collections)
* Specification file editor for mass-changing source packages.


== Continuous rebuilds ==
== Continuous rebuilds ==
'''Problem''': rebuild of new Perl and related packages take a lot of time.
'''Solution''': continuous rebuilds should show broken packages earlier than in annual rebuild.


* [http://www.fedmsg.com/en/latest/ Fedmsg] binding to Perl
* [http://www.fedmsg.com/en/latest/ Fedmsg] binding to Perl
* Use fedmsg to gather updated packages
* Use fedmsg to gather updated packages
** I have already GIT based checker
** [[User:ppisar]] already has GIT based checker
* Use fedmsg to include new packages
* Use fedmsg to include new packages
* Use fedmsg to exclude retired packages
* Use fedmsg to exclude retired packages
* Do periodical scratch boot-strap to evaluate overall package health. Assignee [[user:ppisar]] ✓
** [http://ppisar.fedorapeople.org/perl_rebuild/scratch/latest/index.xhtml logs]
** [http://ppisar.fedorapeople.org/perl_rebuild/scratch/index.xhtml matrix]


== Dependency cycle breaker ==
== Dependency cycle breaker ==
Line 69: Line 105:
* YUM package downloader
* YUM package downloader


== Macros for simplified packaging ==
== Spec file generator ==
'''Problem''': cpanspec is old and not up2date to current guidelines. There is space for improvement of automatic generation of Perl specfiles.


* Version normalizer
'''Solution''': Push changes to cpanspec and continue in work on Perl::Scan package developed by [[User:psabata]].
* perl() wrapper
** This will mask SCL modifications
* Dependency filter abstraction
** One syntax for Fedora and EPEL
** One syntax for SCL
* Provide standard EU::MM, Module::Build, Module::Bild::Tiny, &c. %build and %install implementations
* Standard implementations for CPAN packages
** ''URL''
** ''Sourece0''
** ''%prep''
* Pack files scriptlets editor
** Allow Fatpacker and inc::Module::Build bundler to work on packaged modules
* Mark POD files as documentation
 
== Spec file generator ==


* Fork ''cpan2spec''
* The upstream author of ''cpanspec'' has expressed willingness to incorporate some changes, however a complete rewrite of the utility would be more feasiable long-term
* Static dependency analysis
* Static dependency analysis
** Rewrite Perl-Scan without eval. Assignee: [[User:psabata]]
** Rewrite Perl::Scan without eval. Assignee: [[User:psabata]]
* Dynamic dependency analysis
* Dynamic dependency analysis
* Compare analyses against RPM package
* Compare analyses against RPM package
Line 99: Line 121:
* False provides checker
* False provides checker
** RPM package should not provide privately redefined modules
** RPM package should not provide privately redefined modules
* Converter into collection spec files which understands Perl spec files


== Fedora Upstream monitoring ==
== Fedora Upstream monitoring ==
'''Problem''': Some maintainers don't want to monitor upstream releases. Some upstream releases can't be parsed by Fedora monitoring script. During annual rebuild with new Perl version can updates of modules consume a lot of time.
'''Solution''': 1/ let's try to persuade monitoring system to fix their issues, 2/ see next point


* Monitor upstream on our own if Fedora's system will not be improved
* Monitor upstream on our own if Fedora's system will not be improved
Line 106: Line 132:
* Monitor all RPM packages
* Monitor all RPM packages
** Update them before each mass rebuild
** Update them before each mass rebuild
== [[Features/DevelopersAssistant | Developer Assistant]] ==
* Templates for standard build systems
* Templates for web frameworks


== Upstream monitoring ==
== Upstream monitoring ==


* Not all packages are registered in [[Upstream_release_monitoring | Fedora Upstream Monitoring service]]
* Not all packages are registered in [https://docs.fedoraproject.org/en-US/package-maintainers/Upstream_Release_Monitoring/ Fedora Upstream Monitoring service]
* Not all packagers upgrades
* Not all packagers upgrades
* Monitor upstream on our own
* Monitor upstream on our own
Line 124: Line 145:
** Add the check into ''spectool(1)''. Assignee: [[User:ppisar]]
** Add the check into ''spectool(1)''. Assignee: [[User:ppisar]]
*** Reported against rpmdevtools as a [https://bugzilla.redhat.com/show_bug.cgi?id=1093712 bug #1093712].
*** Reported against rpmdevtools as a [https://bugzilla.redhat.com/show_bug.cgi?id=1093712 bug #1093712].
== [[Features/DevelopersAssistant | Developer Assistant]] ==
* Templates for standard build systems
* Templates for web frameworks


== Perl Next ==
== Perl Next ==


* While perl is a base package, other CPAN code will go into server or workstation. We need to be ready and implement Next requirements
* While perl is a base package, other CPAN code will go into server or workstation. We need to be ready and implement Next requirements. It turned out the the Next was a fiasco. No Perl changes were needed.
* Propose to [https://lists.fedoraproject.org/pipermail/perl-devel/2014-March/085719.html delay perl 5.20 upgrade to Fedora 22]. Assignee: [[User:ppisar]] ✓
* Propose to [https://lists.fedoraproject.org/pipermail/perl-devel/2014-March/085719.html delay perl 5.20 upgrade to Fedora 22]. Assignee: [[User:ppisar]] ✓
* Submit Perl 5.20 change to Fedora 22. Assignee: [[User:jplesnik]] ✓
* Rebuild Perl 5.20. Assignee: [[User:jplesnik]] ✓
* Remove Perl from minimal build root. [https://lists.fedoraproject.org/pipermail/devel/2014-October/203821.html Request]. Assignee: [[User:ppisar]]

Latest revision as of 20:47, 2 October 2021


Version normalization

Problem: sometimes new version could be in form acceptable for CPAN, but not for RPM.

Example: package had version 1.00 and new release is 1.000324. It works for CPAN, where version is just a string, but not for RPM.

Solution: instead of manual change (also provides might be wrong) can be version normalized by Perl itself (module version can do it).

  • After taking over Perl dependency generator ✓
  • Macros for simplified packaging
    • the version translator
      • Perl architecture independent (because of upgrading perl): perl-Fedora-VSP ✓
      • C implementation for build-requires required by perl-srpm-macros
    • the perl() wrapper
    • common snippets for EU::MM, Module::Build, etc.
  • version->parse->normal normalization in macros and generator
  • Normalize RPM package version?

Macros for simplified packaging

Problem: There is still lot of manual work in specfiles. Better macros could improve packaging. Some packagers might discourage changes, because they prefer lot of ifdefs for different EL releases.

Solution: Improvements above could make maintenance easier for future. One macro in macro file can redefine content of all specfiles and rebuild would add all changes.

  • Version normalizer (see first paragraph)
  • perl() wrapper
    • This will mask SCL modifications
  • Dependency filter abstraction
    • One syntax for Fedora and EPEL
    • One syntax for SCL
  • Provide standard EU::MM, Module::Build, Module::Bild::Tiny, &c. %build and %install implementations
  • Standard implementations for CPAN packages
    • URL
    • Sourece0
    • %prep
  • Pack files scriptlets editor
    • Allow Fatpacker and inc::Module::Build bundler to work on packaged modules
  • Mark POD files as documentation

Dependency generator

Problem: dependency generator in RPM has lot of incorrect data, which leads to complicated filtering of provides/requires

Solution: dependency generator will be removed from RPM and owned by Perl team.

Comment: More improvements in main generator are not possible because contyk's smarter scaner has many dependencies. At least contyk's scaner could be used for generating new packages and updates.

  • Take over dependency generator. Assignee: User:jplesnik, User:mmaslano. Packaged as perl-generators. ✓
  • Tests. Assignee: User:jplesnik
  • Support no, base, parent, Moose's with, POE's extensions, Test::Requires, other frameworks, see Module::ScanDeps::scan_chunk()
  • Receive explicit Requires from spec file to merge them with generated ones
  • Refactoring
    • How many dependencies has Module-ScanDeps? Assignee: User:psabata. Module-ScanDeps reads META.yml. It's not useful. ✓

Fedora-Rebuild

Problem: annual Perl rebuild took more than month. No tooling is provided by Fedora nor rpm/yum-tools.

Solution: Fedora::Rebuild is able to count build order of packages based on their dependencies.

Comment: rebuilds of every new release should make easier finding bugs in other packages sooner and make rebuild shorter.

  • Assignee: User:ppisar
  • Optimize pure-perl resolver
    • Finished in git tree. ✓
    • Release new version. ✓
  • Implement hawkey-based resolver
    • XS wrapper for hawkey
    • YUM repository filter for Fedora-Rebuild dependency filter
    • YUM repostory downloader
    • Augment Fedora-Rebuild to download YUM repository, filter it, and do hawkey queries over it.
  • Copr mode
    • Copr is missing way how to retrieve binary package made of only exact source package
    • Select binary packages from Copr's YUM repository
    • Requires YUM downloaded and metadata parser
  • Mock initialization with multiple groups and packages (to support rebuilding collections)
  • Specification file editor for mass-changing source packages.

Continuous rebuilds

Problem: rebuild of new Perl and related packages take a lot of time.

Solution: continuous rebuilds should show broken packages earlier than in annual rebuild.

  • Fedmsg binding to Perl
  • Use fedmsg to gather updated packages
  • Use fedmsg to include new packages
  • Use fedmsg to exclude retired packages
  • Do periodical scratch boot-strap to evaluate overall package health. Assignee user:ppisar

Dependency cycle breaker

  • YUM repository reader
    • repoquery --whatprovides is to slow due to reinitialization
  • Gather boot-strapped dependencies from local SRPMs
  • Use graph algorithms to identify cycles
  • Semi-automated recommendation for cycle break-points

YUM repository parser

  • YUM metadata downloader
  • YUM metadata reader
  • YUM metadata filter/writer
  • YUM package downloader

Spec file generator

Problem: cpanspec is old and not up2date to current guidelines. There is space for improvement of automatic generation of Perl specfiles.

Solution: Push changes to cpanspec and continue in work on Perl::Scan package developed by User:psabata.

  • The upstream author of cpanspec has expressed willingness to incorporate some changes, however a complete rewrite of the utility would be more feasiable long-term
  • Static dependency analysis
  • Dynamic dependency analysis
  • Compare analyses against RPM package
    • Comment annotations in spec
    • Structured build-, run-, test- dependencies
  • Differences calculator
    • Useful when upgrading a package
  • False provides checker
    • RPM package should not provide privately redefined modules
  • Converter into collection spec files which understands Perl spec files

Fedora Upstream monitoring

Problem: Some maintainers don't want to monitor upstream releases. Some upstream releases can't be parsed by Fedora monitoring script. During annual rebuild with new Perl version can updates of modules consume a lot of time.

Solution: 1/ let's try to persuade monitoring system to fix their issues, 2/ see next point

  • Monitor upstream on our own if Fedora's system will not be improved
    • The maintainer wants CPAN wildcard
  • Monitor all RPM packages
    • Update them before each mass rebuild

Upstream monitoring

  • Not all packages are registered in Fedora Upstream Monitoring service
  • Not all packagers upgrades
  • Monitor upstream on our own
    • Use CPAN
    • Adjust versions where done in Fedora
    • File bugs or do the upgrade directly or spam perl-devel mailing list
  • CPAN mirrors provides signed digests of CPAN distributions
    • Verify them against Fedora look-aside cache
    • Add the check into spectool(1). Assignee: User:ppisar

Developer Assistant

  • Templates for standard build systems
  • Templates for web frameworks

Perl Next

  • While perl is a base package, other CPAN code will go into server or workstation. We need to be ready and implement Next requirements. It turned out the the Next was a fiasco. No Perl changes were needed.
  • Propose to delay perl 5.20 upgrade to Fedora 22. Assignee: User:ppisar
  • Submit Perl 5.20 change to Fedora 22. Assignee: User:jplesnik ✓
  • Rebuild Perl 5.20. Assignee: User:jplesnik
  • Remove Perl from minimal build root. Request. Assignee: User:ppisar