From Fedora Project Wiki
(→‎Fedora-Rebuild: Mock initialization with multiple groups and packages added)
(Redirect Package Maintainer wiki links to docs.fp.o)
 
(6 intermediate revisions by 2 users not shown)
Line 8: Line 8:
'''Solution''': instead of manual change (also provides might be wrong) can be version normalized by Perl itself (module version can do it).
'''Solution''': instead of manual change (also provides might be wrong) can be version normalized by Perl itself (module version can do it).


* After taking over Perl dependency generator
* After taking over Perl dependency generator ✓
* Macros for simplified packaging
* Macros for simplified packaging
** the version translator
** the version translator
*** Perl architecture independent (because of upgrading perl): perl-Fedora-VSP ✓
*** C implementation for build-requires required by perl-srpm-macros
** the ''perl()'' wrapper
** the ''perl()'' wrapper
** common snippets for EU::MM, Module::Build, etc.
** common snippets for EU::MM, Module::Build, etc.
* <code>version->parse->normal</code> normalization in macros and generator
* <code>version->parse->normal</code> normalization in macros and generator
* Normalize RPM package version?
* Normalize RPM package version?
** Teach [[Upstream_release_monitoring | Fedora upstream monitoring]] ''cnucnu(1)''
** Teach [https://docs.fedoraproject.org/en-US/package-maintainers/Upstream_Release_Monitoring/ Fedora upstream monitoring] ''cnucnu(1)''


== Macros for simplified packaging ==
== Macros for simplified packaging ==
Line 72: Line 74:
** Requires YUM downloaded and metadata parser
** Requires YUM downloaded and metadata parser
* Mock initialization with multiple groups and packages (to support rebuilding collections)
* Mock initialization with multiple groups and packages (to support rebuilding collections)
* Specification file editor for mass-changing source packages.


== Continuous rebuilds ==
== Continuous rebuilds ==
Line 118: Line 121:
* False provides checker
* False provides checker
** RPM package should not provide privately redefined modules
** RPM package should not provide privately redefined modules
* Converter into collection spec files which understands Perl spec files


== Fedora Upstream monitoring ==
== Fedora Upstream monitoring ==
Line 131: Line 135:
== Upstream monitoring ==
== Upstream monitoring ==


* Not all packages are registered in [[Upstream_release_monitoring | Fedora Upstream Monitoring service]]
* Not all packages are registered in [https://docs.fedoraproject.org/en-US/package-maintainers/Upstream_Release_Monitoring/ Fedora Upstream Monitoring service]
* Not all packagers upgrades
* Not all packagers upgrades
* Monitor upstream on our own
* Monitor upstream on our own
Line 149: Line 153:
== Perl Next ==
== Perl Next ==


* While perl is a base package, other CPAN code will go into server or workstation. We need to be ready and implement Next requirements
* While perl is a base package, other CPAN code will go into server or workstation. We need to be ready and implement Next requirements. It turned out the the Next was a fiasco. No Perl changes were needed.
* Propose to [https://lists.fedoraproject.org/pipermail/perl-devel/2014-March/085719.html delay perl 5.20 upgrade to Fedora 22]. Assignee: [[User:ppisar]] ✓
* Propose to [https://lists.fedoraproject.org/pipermail/perl-devel/2014-March/085719.html delay perl 5.20 upgrade to Fedora 22]. Assignee: [[User:ppisar]] ✓
* Submit Perl 5.20 change to Fedora 22. Assignee: [[User:jplesnik]]
* Submit Perl 5.20 change to Fedora 22. Assignee: [[User:jplesnik]] ✓
* Rebuild Perl 5.20. Assignee: [[User:jplesnik]] ✓
* Remove Perl from minimal build root. [https://lists.fedoraproject.org/pipermail/devel/2014-October/203821.html Request]. Assignee: [[User:ppisar]]

Latest revision as of 20:47, 2 October 2021


Version normalization

Problem: sometimes new version could be in form acceptable for CPAN, but not for RPM.

Example: package had version 1.00 and new release is 1.000324. It works for CPAN, where version is just a string, but not for RPM.

Solution: instead of manual change (also provides might be wrong) can be version normalized by Perl itself (module version can do it).

  • After taking over Perl dependency generator ✓
  • Macros for simplified packaging
    • the version translator
      • Perl architecture independent (because of upgrading perl): perl-Fedora-VSP ✓
      • C implementation for build-requires required by perl-srpm-macros
    • the perl() wrapper
    • common snippets for EU::MM, Module::Build, etc.
  • version->parse->normal normalization in macros and generator
  • Normalize RPM package version?

Macros for simplified packaging

Problem: There is still lot of manual work in specfiles. Better macros could improve packaging. Some packagers might discourage changes, because they prefer lot of ifdefs for different EL releases.

Solution: Improvements above could make maintenance easier for future. One macro in macro file can redefine content of all specfiles and rebuild would add all changes.

  • Version normalizer (see first paragraph)
  • perl() wrapper
    • This will mask SCL modifications
  • Dependency filter abstraction
    • One syntax for Fedora and EPEL
    • One syntax for SCL
  • Provide standard EU::MM, Module::Build, Module::Bild::Tiny, &c. %build and %install implementations
  • Standard implementations for CPAN packages
    • URL
    • Sourece0
    • %prep
  • Pack files scriptlets editor
    • Allow Fatpacker and inc::Module::Build bundler to work on packaged modules
  • Mark POD files as documentation

Dependency generator

Problem: dependency generator in RPM has lot of incorrect data, which leads to complicated filtering of provides/requires

Solution: dependency generator will be removed from RPM and owned by Perl team.

Comment: More improvements in main generator are not possible because contyk's smarter scaner has many dependencies. At least contyk's scaner could be used for generating new packages and updates.

  • Take over dependency generator. Assignee: User:jplesnik, User:mmaslano. Packaged as perl-generators. ✓
  • Tests. Assignee: User:jplesnik
  • Support no, base, parent, Moose's with, POE's extensions, Test::Requires, other frameworks, see Module::ScanDeps::scan_chunk()
  • Receive explicit Requires from spec file to merge them with generated ones
  • Refactoring
    • How many dependencies has Module-ScanDeps? Assignee: User:psabata. Module-ScanDeps reads META.yml. It's not useful. ✓

Fedora-Rebuild

Problem: annual Perl rebuild took more than month. No tooling is provided by Fedora nor rpm/yum-tools.

Solution: Fedora::Rebuild is able to count build order of packages based on their dependencies.

Comment: rebuilds of every new release should make easier finding bugs in other packages sooner and make rebuild shorter.

  • Assignee: User:ppisar
  • Optimize pure-perl resolver
    • Finished in git tree. ✓
    • Release new version. ✓
  • Implement hawkey-based resolver
    • XS wrapper for hawkey
    • YUM repository filter for Fedora-Rebuild dependency filter
    • YUM repostory downloader
    • Augment Fedora-Rebuild to download YUM repository, filter it, and do hawkey queries over it.
  • Copr mode
    • Copr is missing way how to retrieve binary package made of only exact source package
    • Select binary packages from Copr's YUM repository
    • Requires YUM downloaded and metadata parser
  • Mock initialization with multiple groups and packages (to support rebuilding collections)
  • Specification file editor for mass-changing source packages.

Continuous rebuilds

Problem: rebuild of new Perl and related packages take a lot of time.

Solution: continuous rebuilds should show broken packages earlier than in annual rebuild.

  • Fedmsg binding to Perl
  • Use fedmsg to gather updated packages
  • Use fedmsg to include new packages
  • Use fedmsg to exclude retired packages
  • Do periodical scratch boot-strap to evaluate overall package health. Assignee user:ppisar

Dependency cycle breaker

  • YUM repository reader
    • repoquery --whatprovides is to slow due to reinitialization
  • Gather boot-strapped dependencies from local SRPMs
  • Use graph algorithms to identify cycles
  • Semi-automated recommendation for cycle break-points

YUM repository parser

  • YUM metadata downloader
  • YUM metadata reader
  • YUM metadata filter/writer
  • YUM package downloader

Spec file generator

Problem: cpanspec is old and not up2date to current guidelines. There is space for improvement of automatic generation of Perl specfiles.

Solution: Push changes to cpanspec and continue in work on Perl::Scan package developed by User:psabata.

  • The upstream author of cpanspec has expressed willingness to incorporate some changes, however a complete rewrite of the utility would be more feasiable long-term
  • Static dependency analysis
  • Dynamic dependency analysis
  • Compare analyses against RPM package
    • Comment annotations in spec
    • Structured build-, run-, test- dependencies
  • Differences calculator
    • Useful when upgrading a package
  • False provides checker
    • RPM package should not provide privately redefined modules
  • Converter into collection spec files which understands Perl spec files

Fedora Upstream monitoring

Problem: Some maintainers don't want to monitor upstream releases. Some upstream releases can't be parsed by Fedora monitoring script. During annual rebuild with new Perl version can updates of modules consume a lot of time.

Solution: 1/ let's try to persuade monitoring system to fix their issues, 2/ see next point

  • Monitor upstream on our own if Fedora's system will not be improved
    • The maintainer wants CPAN wildcard
  • Monitor all RPM packages
    • Update them before each mass rebuild

Upstream monitoring

  • Not all packages are registered in Fedora Upstream Monitoring service
  • Not all packagers upgrades
  • Monitor upstream on our own
    • Use CPAN
    • Adjust versions where done in Fedora
    • File bugs or do the upgrade directly or spam perl-devel mailing list
  • CPAN mirrors provides signed digests of CPAN distributions
    • Verify them against Fedora look-aside cache
    • Add the check into spectool(1). Assignee: User:ppisar

Developer Assistant

  • Templates for standard build systems
  • Templates for web frameworks

Perl Next

  • While perl is a base package, other CPAN code will go into server or workstation. We need to be ready and implement Next requirements. It turned out the the Next was a fiasco. No Perl changes were needed.
  • Propose to delay perl 5.20 upgrade to Fedora 22. Assignee: User:ppisar
  • Submit Perl 5.20 change to Fedora 22. Assignee: User:jplesnik ✓
  • Rebuild Perl 5.20. Assignee: User:jplesnik
  • Remove Perl from minimal build root. Request. Assignee: User:ppisar