This file briefly describes the requirements for enabling or creating a custom BLIS addon.
Simply put, an addon in BLIS provides additional APIs, operations, and/or implementations that may be useful to certain users. An addon can be thought of as a standalone extension of BLIS that does not depend on any other addon, although addons may utilize existing functionality or kernels within the core framework.
By definition, an addon should never provide APIs that conflict with the interfaces that belong to either the typed API or the object API. Thus, you'll never have to worry about a properly constructed (and properly functioning) addon interfering with or otherwise changing core BLIS functionality.
How does an addon differ from a sandbox? Great question!
Sometimes you want to include additional BLIS-like functionality that does
not relate directly to gemm or any other BLIS operation.
(By contrast, a sandbox requires you to implement gemm whether you want
to or not.)
Furthermore, you may wish to enable multiple addons simultaneously.
(By contrast, only one sandbox may be enabled at a time.)
Thus, the addon feature provides additional flexibility to some
users in a way that sandboxes cannot, while still providing many of the
conveniences of sandboxes.
To enable an existing addon at configure-time, you simply specify it as an
option to configure. Either of the following usages are accepted:
$ ./configure --enable-addon=foobar auto
$ ./configure -a foobar auto
Here, we tell configure that we want to use the foobar addon, which
corresponds to a subdirectory of the addon directory named foobar.
(Reminder: the auto argument is the configuration target and
unrelated to addons.)
You may also enable multiple addons within the same build of BLIS:
$ ./configure -a foobar -a thing1 -a thing2 auto
Note that the default behavior of configure is that no addons are enabled.
As configure runs, you should get output that includes lines
similar to:
configure: configuring with addons:
configure: addon/foobar
configure: addon/thing1
configure: addon/thing2
And when you build BLIS, the addon source code will be among the last files to be compiled:
Compiling obj/haswell/addon/foobar/foobar.o ('haswell' CFLAGS for addons)
Compiling obj/haswell/addon/thing1/thing1.o ('haswell' CFLAGS for addons)
Compiling obj/haswell/addon/thing1/thing1_api.o ('haswell' CFLAGS for addons)
Compiling obj/haswell/addon/thing2/thing2_api.o ('haswell' CFLAGS for addons)
...
That's it! After the BLIS library is built, it will contain your chosen
addons. You can always confirm this by using nm to confirm the presence
of your API symbols:
$ nm lib/haswell/libblis.a | grep foobar
foobar.o:
0000000000000000 T foobar
Please follow these guidelines for the best developer experience when creating addons.
-
As with sandboxes, you don't need to worry about creating makefiles. The BLIS build system will take care of this for you. :) By configuring BLIS with an addon enabled,
makewill scan your addon subdirectory and compile all of its source code using similar compilation rules as were used for the rest of the framework. In addition, the compilation command line will automatically contain one-I<includepath>option for every subdirectory in your addon, so it doesn't matter where in your addon directory hierarchy you place your header files -- they will be found! -
We recommend that you write your addon in C99. While you may use C++11 to implement your addon, you should provide a C99 wrapper API to your implementation so that others can interface with it. There is no guarantee that the end-user will be using a C++11 compiler, and therefore you should limit the definitions in your addon header to those that are C99 compliant. If you write your addon in C++11, you must use one of the BLIS-approved file extensions for your source files (
.cc,.cpp,.cxx) and your local header files (.hh,.hpp,.hxx). Note thatblis.halready contains all of its definitions inside of anextern "C"block, so you should be able to#include "blis.h"from your C++11 source code without any issues. -
All of your code related to the addon should reside within the named addon directory, or some subdirectory therein. If your addon requires new kernels, you should add kernel source code to an appropriate microarchitecture-specific subdirectory within the top-level
kernelsdirectory so that they are compiled with the correct microarchitecture-specific optimization flags. -
If your addon is named
foobar, the BLIS build system will expect to find a header calledfoobar.hsomewhere in theaddon/foobardirectory (or one of its subdirectories). Thisfoobar.hheader will automatically be inlined into the monolithicblis.hheader that is produced by the BLIS build system.foobar.hmay#includeother local headers, each of which will also (recursively) get inlined intoblis.h. However, you may choose to omit some local addon headers fromfoobar.h.You might do this, for example, because those headers define things that are not needed in order for the end user to call your addon code. -
Your addon APIs will always be available within static library builds of BLIS, but if you want your addon APIs to be exported as public APIs within shared library builds of BLIS, you'll need to annotate the prototypes accordingly. (BLIS makes its shared library symbols private by default; this allows us to export only those functions that we consider to be part of the public APIs.) This annotation can be done by prefixing function prototypes with the
BLIS_EXPORT_ADDONmacro as follows:
BLIS_EXPORT_ADDON void foobar_calc( void* a, void* b );-
Do not define any symbols in your addon that conflict with any symbols within the core framework. For example, don't define a function called
bli_copym()in your addon since that function is already defined within BLIS. -
Do not define any symbols in your addon that conflict with any symbols within the C99 standard libraries/headers. For example, don't define a function called
printf()since that function is already defined within the C99 standard library. -
Try to not define any symbols in your addon that conflict with symbols in any other addon, unless your addon is meant to serve as an alternative to the conflicting addon, in which case conflicting symbol names is okay (since you will presumably never build with both addons enabled).
-
When choosing names for your addon files, avoid source filenames that already exist within BLIS. For example, don't name one of your files
bli_obj.csince that file would compile intobli_obj.o, which will have already been placed into the library by the build system. -
Similarly, avoid header filenames that already exist within BLIS or C99. For example, don't name one of your header files
bli_obj.hsince that file already exists in BLIS. Also, don't name one of your header filesmath.hsince that name would conflict with themath.hdefined by C99. (This also means you shouldn't name your addonmathsince normally that name would require that you provide amath.hheader inside the addon directory.)
If you follow these rules, you will be much more likely to have a pleasant experience integrating your BLIS addon into the larger framework.
Notice that the BLIS addons are limited in what they can accomplish. Generally speaking, addons cannot change existing implementations within BLIS. Instead, addons aim to provide a way to quickly augment BLIS with additional bundles of code that extend BLIS's set of functionality in some interesting way. If you want to define new BLAS-like functions, but don't know where to start, creating a new addon is an appropriate place to start experimenting. If you want to change or refactor existing BLIS code, an addon is probably not suited for your needs.
Another important limitation is the fact that the build system currently uses
"framework CFLAGS" when compiling the addon source files. These are the same
CFLAGS used when compiling general framework source code,
# Example framework CFLAGS used by 'haswell' sub-configuration
-O2 -Wall -Wno-unused-function -Wfatal-errors -fPIC -std=c99
-D_POSIX_C_SOURCE=200112L -Iinclude/haswell -I./frame/3/
-I./frame/1m/ -I./frame/1f/ -I./frame/1/ -I./frame/include
-DBLIS_VERSION_STRING=\"0.8.1-195\" -fvisibility=hidden
which are likely more general-purpose than the CFLAGS used for, say,
optimized kernels or even reference kernels:
# Example optimized kernel CFLAGS used by 'haswell' sub-configuration
-O3 -fomit-frame-pointer -mavx2 -mfma -mfpmath=sse -march=haswell -Wall
-Wno-unused-function -Wfatal-errors -fPIC -std=c99 -D_POSIX_C_SOURCE=200112L
-Iinclude/haswell -I./frame/3/ -I./frame/1m/ -I./frame/1f/ -I./frame/1/
-I./frame/include -DBLIS_VERSION_STRING=\"0.8.1-195\" -fvisibility=hidden
(To see precisely which flags are being employed for any given file, enable
verbosity at compile-time via make V=1.) Compiling addons with these more
versatile CFLAGS compiler options means that we only need to compile one
instance of each addon source file, even when targeting multiple
configurations (for example, via ./configure x86_64). However, it also means
that addons are not ideal for microkernels, as they sometimes need additional
compiler flags in order to
yield the highest performance. If you have a new microkernel you would like to
use within an addon, you can always develop it within that addon. However,
once it is stable and ready for use by others, it's best to move the kernel(s)
to the appropriate microarchitecture-specific subdirectory of the kernels
directory the kernel(s). This will allow the kernel to be compiled with the
appropriate microarchitecture-specific compiler flags.
Please see the
Configuration Guide
for more details, and when in doubt, please don't be shy about seeking
guidance from BLIS developers by opening a
new issue or sending a message to the
blis-devel mailing list.
Notwithstanding these limitations, hopefully you still find BLIS addons useful!
- None yet.
If you encounter any problems, please open a new issue on GitHub.
If you are unsure about how something works, you can still open an issue. Or, you can send a message to blis-devel mailing list.