From Fedora Project Wiki
(→‎BLAS/LAPACK implementations: note that this doesn't work)
mNo edit summary
(11 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Draft}}
{{Draft}}
= Introduction =
= Introduction =


Besides being implementations, BLAS and LAPACK are also API standards for basic linear algebra operations (such as vector and matrix multiplication).
Besides being implementations, BLAS and LAPACK are also API standards for basic linear algebra operations (such as vector and matrix multiplication). Many implementations of these API exist. The reference implementation of BLAS and LAPACK from Netlib is very stable but is not as fast as optimized ones such as ATLAS and OpenBLAS.


Many implementations of these API exist. The reference implementation of BLAS and LAPACK from netlib is very stable but is not as fast as optimized ones such as ATLAS and OpenBLAS.
Implementations of BLAS:


Implementations of BLAS:
* blas - Reference implementation from Netlib.
* atlas - Automatically Tuned Linear Algebra Software.
* blis - BLAS-like Library Instantiation Software framework.
* openblas - OpenBLAS, an optimized BLAS based on GotoBLAS2.


*    blas - Reference implementation from netlib
Both BLIS and OpenBLAS provide several flavors: a sequential version, a threaded one, and another with OpenMP support (all of them with or without support for 64-bit integers).
*    atlas - Automatically Tuned Linear Algebra Software
*    openblas - OpenBLAS, an optimized BLAS based on GotoBLAS2


Implementations of LAPACK:
Implementations of LAPACK:


*   lapack - Reference implementation from netlib
* lapack - Reference implementation from Netlib.
*   ATLAS and OpenBLAS both provide an optimized subset of LAPACK
* ATLAS, BLIS and OpenBLAS provide an optimized subset of LAPACK.


Due to implementation differences, it is important that all components of a particular software stack link to the same BLAS/LAPACK implementation. Also, users may want to choose a particular implementation that works best for them at run time.  This guideline gives a structure that can enforce the first while allowing the second.
Due to implementation differences, it is important that all components of a particular software stack link to the same BLAS/LAPACK implementation. Also, users may want to choose a particular implementation that works best for them at run time.  This guideline gives a structure that can enforce the first while allowing the second, as well as providing a transparent fallback mechanism to Netlib's reference implementation for those symbols not included in the selected backend.


= BLAS/LAPACK implementations =
= BLAS/LAPACK wrapper =


[https://www.mpi-magdeburg.mpg.de/projects/flexiblas FlexiBLAS] is a framework that wraps both BLAS and LAPACK APIs in a single library. BLAS/LAPACK consumers must link against FlexiBLAS, and this wrapper is able to redirect calls to a selected optimized backend with negligible overhead. It also provides transparent fallback to Netlib's reference implementation if a certain symbol is not present in the selected backend. These are the main features:


{{admon/important|This doesn't work.  We need to build versions of the libraries with the blas/lapack sonames and symbols.}}
* Provides a 100% BLAS and LAPACK compatible ABI/API, with interfaces for both 32- and 64-bit integers.
* Runtime exchangeable BLAS and LAPACK backend without recompilation via an environment variable.
Implementations of a BLAS and/or LAPACK library will provide relative symbolic links in %{_libdir}/IMPLEMENTATION-NAME/ of libblas.so.3 and/or liblapack.so.3 as appropriate pointing to the actual implementation library.  Each implementation may have multiple IMPLEMENTATION-NAMEs, e.g. for serial and parallel versions.
* Integration of user-owned BLAS libraries without administrator privileges, even in system-wide installed programs.
* Works with OpenBLAS, ATLAS, Intel MKL, ACML, ...
* Flexible per-system/user/host configuration files.
* Basic profiling support.


For example, atlas would do:
FlexiBLAS provides a common interface, as well as fallbacks to Netlib's reference implementation:


  %install
  %files netlib
  ...
  %{_libdir}/flexiblas.so.x
  ln -s ../libsatlas.so.3 %{buildroot}%{_libdir}/%{name}-serial/libblas.so.3
  %{_libdir}/flexiblas/libflexiblas_netlib.so
  ln -s ../libsatlas.so.3 %{buildroot}%{_libdir}/%{name}-serial/liblapack.so.3
%{_libdir}/flexiblas/libflexiblas_fallback_lapack.so
  ln -s ../libtatlas.so.3 %{buildroot}%{_libdir}/%{name}-threaded/libblas.so.3
   
  ln -s ../libtatlas.so.3 %{buildroot}%{_libdir}/%{name}-threaded/liblapack.so.3
%files netlib64
%{_libdir}/flexiblas64.so.x
  %{_libdir}/flexiblas64/libflexiblas_netlib.so
  %{_libdir}/flexiblas64/libflexiblas_fallback_lapack.so


If ILP64 (64-bit integer) implementations are available, they should be provided as well.  For openblas:
and, finally, several wrappers for different implementations under different names:


  %install
  %files atlas
  ...
  %{_libdir}/flexiblas/libflexiblas_atlas.so
  ln -s ../libopenblas64.so.0 %{buildroot}%{_libdir}/%[name}-serial/libblas64.so.3
  ln -s ../libopenblas64.so.0 %{buildroot}%{_libdir}/%{name}-serial/liblapack64.so.3
  %files blis-serial
  ln -s ../libopenblas64_.so.0 %{buildroot}%{_libdir}/%{name}-serial/libblas64_.so.3
%{_libdir}/flexiblas/libflexiblas_blis-serial.so
  ln -s ../libopenblas64_.so.0 %{buildroot}%{_libdir}/%{name}-serial/liblapack64_.so.3
%files blis-threads
%{_libdir}/flexiblas/libflexiblas_blis-threads.so
   
%files blis-openmp
%{_libdir}/flexiblas/libflexiblas_blis-openmp.so
%files openblas-serial
%{_libdir}/flexiblas/libflexiblas_openblas-serial.so
   
%files openblas-threads
%{_libdir}/flexiblas/libflexiblas_openblas-threads.so
%files openblas-openmp
%{_libdir}/flexiblas/libflexiblas_openblas-openmp.so
   
plus the 64-bit versions for BLIS and OpenBLAS under %{_libdir}/flexiblas64 in 64-bit architectures. These are sub-packaged separately to avoid pulling the whole stack of implementations. The netlib and netlib64 sub-packages require a default implementation that is currently set to openblas-serial:


== System level BLAS/LAPACK selection ==
%global default_backend openblas-serial
%package        netlib
Requires:      %{name}%{?_isa} = %{version}-%{release}
Requires:      %{name}-%{default_backend}%{?_isa} = %{version}-%{release}
%package        netlib64
Requires:      %{name}%{?_isa} = %{version}-%{release}
Requires:      %{name}-%{default_backend}64%{?_isa} = %{version}-%{release}


To allow system level selection of the desired BLAS/LAPACK implementation, alternatives are used, selecting which path is desired for /etc/ld.so.conf.d/blas-%{arch}.conf. 
= Packaging guidelines =


<pre>
== BLAS/LAPACK implementations ==
Requires(post): %{_sbindir}/update-alternatives
Requires(postun): %{_sbindir}/update-alternatives


%install
Implementations of BLAS and/or LAPACK available in Fedora do not need to change anything.
...
mkdir -p %{buildroot}%{_sysconfdir}/ld.so.conf.d
touch %{buildroot}%{_sysconfdir}/ld.so.conf.d/blas-%{_arch}.conf
echo %{_libdir}/%{name}-serial > %{buildroot}%{_sysconfdir}/ld.so.conf.d/blas-%{_arch}.conf-%{name}-serial


%post
== BLAS/LAPACK dependent packages ==
%{_sbindir}/update-alternatives --install %{_sysconfdir}/ld.so.conf.d/blas-%{_arch}.conf \
  blas %{_sysconfdir}/ld.so.conf.d/blas-%{_arch}.conf-%{name}-serial 60
/sbin/ldconfig


%postun
Consumers of any subset of BLAS and/or LAPACK must compile against FlexiBLAS:
if [ $1 -eq 0 ] ; then
  %{_sbindir}/update-alternatives --remove blas %{_sysconfdir}/ld.so.conf.d/blas-%{_arch}.conf-%{name}-serial
fi
/sbin/ldconfig
</pre>


Current established priorities are 30 for atlas and 60 for openblas.
BuildRequires: pkgconfig(flexiblas)


Also, we provide libblas/liblapack:
if the package requires the 32-bit interface (this is the most common case), or a conditional BuildRequires such as


  %if %{_lib} == "lib64"
  %if 0%{?__isa_bits} == 64
  Provides: libblas.so.3()(64bit)
  BuildRequires: pkgconfig(flexiblas64)
Provides: liblapack.so.3()(64bit)
  %else
  %else
  Provides: libblas.so.3
  BuildRequires: pkgconfig(flexiblas)
Provides: liblapack.so.3
  %endif
  %endif


== User level BLAS/LAPACK selection ==
if the package uses the 64-bit interface when available. Another possibility would be to build 64-bit versions for 64-bit architectures as subpackages. However, note that if the package '''only''' supports this 64-bit interface, then 32-bit architectures must be excluded.


Environment modules are used to provide user level selection of the desired BLAS/LAPACK implementation.  For each IMPLEMENTATION-NAME, the implementation package will provide an environment module, e.g.:
In any case, by compiling against FlexiBLAS, they will automatically use the system-wide default backend, and will benefit from any other system/user/host-selected optimized subset of routines.


#%Module 1.0
= Backend selection =
# ATLAS module for loading serial atlas library
conflict                blas
prepend-path            LD_LIBRARY_PATH /usr/lib64/atlas-serial


as %{_datadir}/modulefiles/blas/atlas-serial.  The implementation must then require an environment module implementation:
== System-level selection ==


Requires: environment(modules)
A package compiled against FlexiBLAS pulls out the corresponding flexiblas-netlib(64) subpackage, which in turn requires the default optimized backend, e.g., flexiblas-openblas-serial(64). This is set via the "default=IMPLEMENTATION-NAME" key (by default, default=openblas-serial), present in the main configuration file shipped in the main subpackages:


Because many (all except the reference?) BLAS/LAPACK implementations are combined BLAS/LAPACK libraries, we do not support separate specification of BLAS and LAPACK, only combinations that are explicitly known to work.
  %files netlib
 
  %config(noreplace) %{_sysconfdir}/flexiblasrc
== Consistent LAPACK versions ==
   
 
  %files netlib64
Most optimized BLAS implementations also offer a subset of optimized LAPACK functions. They must then fill out the rest of the LAPACK API from the reference netlib LAPACK, either via static LAPACK library or a bundled version. The reference LAPACK interface is slowly changing so it is important that the LAPACK API version be consistent across all LAPCK implementations on a specific release.  This is most easily handled by linking to the static LAPACK reference library.
  %config(noreplace) %{_sysconfdir}/flexiblas64rc
 
= BLAS/LAPACK dependent packages =
 
== Generic Usage ==
If the BLAS/LAPACK consumer does not require special compile time configuration for different BLAS/LAPACK implementations, it should simply build against the reference implementation:
 
  BuildRequires: blas-devel
  BuildRequires: lapack-devel


so that they end up linking to libblas.so.3 and liblapack.so.3, thus allowing the implementation to be switched at run time.
To allow system-level selection of other BLAS/LAPACK implementations, more backends must be installed in the first place (e.g., flexiblas-atlas, flexiblas-blis-serial...), and then they can be swapped system-wide via the flexiblas CLI tool, or just by modifying the "default" key in the configuration file by hand.


== Compile time configuration ==
== User-level selection ==


If the consumer requires special configuration for different implementations, it should provide versions compiled with each implementation.
Persistent user-level selection of system-provided BLAS/LAPACK implementations can be done via the CLI tool:


= End-User Documentation =
$ flexiblas set IMPLEMENTATION-NAME
$ flexiblas64 set IMPLEMENTATION-NAME


End users will load the implementation they desire with:
provided the sub-package for IMPLEMENTATION-NAME is installed.


module load blas/IMPLEMENTATION-NAME
Non-persistent user-level selection can be triggered via an environment variable:


TODO - choose a distribution default?
$ FLEXIBLAS=IMPLEMENTATION-NAME ./yourapp
$ FLEXIBLAS64=IMPLEMENTATION-NAME ./yourapp64


User-level selection of user-owned BLAS/LAPACK libraries can be achieved just by changing IMPLEMENTATION-NAME with a path to any custom BLAS/LAPACK-compatible library in the examples above.


[[Category:Packaging guidelines drafts]]
[[Category:Packaging guidelines drafts]]

Revision as of 15:43, 30 June 2020

Warning.png
This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Introduction

Besides being implementations, BLAS and LAPACK are also API standards for basic linear algebra operations (such as vector and matrix multiplication). Many implementations of these API exist. The reference implementation of BLAS and LAPACK from Netlib is very stable but is not as fast as optimized ones such as ATLAS and OpenBLAS.

Implementations of BLAS:

  • blas - Reference implementation from Netlib.
  • atlas - Automatically Tuned Linear Algebra Software.
  • blis - BLAS-like Library Instantiation Software framework.
  • openblas - OpenBLAS, an optimized BLAS based on GotoBLAS2.

Both BLIS and OpenBLAS provide several flavors: a sequential version, a threaded one, and another with OpenMP support (all of them with or without support for 64-bit integers).

Implementations of LAPACK:

  • lapack - Reference implementation from Netlib.
  • ATLAS, BLIS and OpenBLAS provide an optimized subset of LAPACK.

Due to implementation differences, it is important that all components of a particular software stack link to the same BLAS/LAPACK implementation. Also, users may want to choose a particular implementation that works best for them at run time. This guideline gives a structure that can enforce the first while allowing the second, as well as providing a transparent fallback mechanism to Netlib's reference implementation for those symbols not included in the selected backend.

BLAS/LAPACK wrapper

FlexiBLAS is a framework that wraps both BLAS and LAPACK APIs in a single library. BLAS/LAPACK consumers must link against FlexiBLAS, and this wrapper is able to redirect calls to a selected optimized backend with negligible overhead. It also provides transparent fallback to Netlib's reference implementation if a certain symbol is not present in the selected backend. These are the main features:

  • Provides a 100% BLAS and LAPACK compatible ABI/API, with interfaces for both 32- and 64-bit integers.
  • Runtime exchangeable BLAS and LAPACK backend without recompilation via an environment variable.
  • Integration of user-owned BLAS libraries without administrator privileges, even in system-wide installed programs.
  • Works with OpenBLAS, ATLAS, Intel MKL, ACML, ...
  • Flexible per-system/user/host configuration files.
  • Basic profiling support.

FlexiBLAS provides a common interface, as well as fallbacks to Netlib's reference implementation:

%files netlib
%{_libdir}/flexiblas.so.x
%{_libdir}/flexiblas/libflexiblas_netlib.so
%{_libdir}/flexiblas/libflexiblas_fallback_lapack.so

%files netlib64
%{_libdir}/flexiblas64.so.x
%{_libdir}/flexiblas64/libflexiblas_netlib.so
%{_libdir}/flexiblas64/libflexiblas_fallback_lapack.so

and, finally, several wrappers for different implementations under different names:

%files atlas
%{_libdir}/flexiblas/libflexiblas_atlas.so

%files blis-serial
%{_libdir}/flexiblas/libflexiblas_blis-serial.so

%files blis-threads
%{_libdir}/flexiblas/libflexiblas_blis-threads.so

%files blis-openmp
%{_libdir}/flexiblas/libflexiblas_blis-openmp.so

%files openblas-serial
%{_libdir}/flexiblas/libflexiblas_openblas-serial.so

%files openblas-threads
%{_libdir}/flexiblas/libflexiblas_openblas-threads.so

%files openblas-openmp
%{_libdir}/flexiblas/libflexiblas_openblas-openmp.so

plus the 64-bit versions for BLIS and OpenBLAS under %{_libdir}/flexiblas64 in 64-bit architectures. These are sub-packaged separately to avoid pulling the whole stack of implementations. The netlib and netlib64 sub-packages require a default implementation that is currently set to openblas-serial:

%global default_backend openblas-serial

%package        netlib
Requires:       %{name}%{?_isa} = %{version}-%{release}
Requires:       %{name}-%{default_backend}%{?_isa} = %{version}-%{release}

%package        netlib64
Requires:       %{name}%{?_isa} = %{version}-%{release}
Requires:       %{name}-%{default_backend}64%{?_isa} = %{version}-%{release}

Packaging guidelines

BLAS/LAPACK implementations

Implementations of BLAS and/or LAPACK available in Fedora do not need to change anything.

BLAS/LAPACK dependent packages

Consumers of any subset of BLAS and/or LAPACK must compile against FlexiBLAS:

BuildRequires: pkgconfig(flexiblas)

if the package requires the 32-bit interface (this is the most common case), or a conditional BuildRequires such as

%if 0%{?__isa_bits} == 64
BuildRequires: pkgconfig(flexiblas64)
%else
BuildRequires: pkgconfig(flexiblas)
%endif

if the package uses the 64-bit interface when available. Another possibility would be to build 64-bit versions for 64-bit architectures as subpackages. However, note that if the package only supports this 64-bit interface, then 32-bit architectures must be excluded.

In any case, by compiling against FlexiBLAS, they will automatically use the system-wide default backend, and will benefit from any other system/user/host-selected optimized subset of routines.

Backend selection

System-level selection

A package compiled against FlexiBLAS pulls out the corresponding flexiblas-netlib(64) subpackage, which in turn requires the default optimized backend, e.g., flexiblas-openblas-serial(64). This is set via the "default=IMPLEMENTATION-NAME" key (by default, default=openblas-serial), present in the main configuration file shipped in the main subpackages:

%files netlib
%config(noreplace) %{_sysconfdir}/flexiblasrc

%files netlib64
%config(noreplace) %{_sysconfdir}/flexiblas64rc

To allow system-level selection of other BLAS/LAPACK implementations, more backends must be installed in the first place (e.g., flexiblas-atlas, flexiblas-blis-serial...), and then they can be swapped system-wide via the flexiblas CLI tool, or just by modifying the "default" key in the configuration file by hand.

User-level selection

Persistent user-level selection of system-provided BLAS/LAPACK implementations can be done via the CLI tool:

$ flexiblas set IMPLEMENTATION-NAME
$ flexiblas64 set IMPLEMENTATION-NAME

provided the sub-package for IMPLEMENTATION-NAME is installed.

Non-persistent user-level selection can be triggered via an environment variable:

$ FLEXIBLAS=IMPLEMENTATION-NAME ./yourapp
$ FLEXIBLAS64=IMPLEMENTATION-NAME ./yourapp64

User-level selection of user-owned BLAS/LAPACK libraries can be achieved just by changing IMPLEMENTATION-NAME with a path to any custom BLAS/LAPACK-compatible library in the examples above.