Switch between MKL and OpenBLAS at runtime

[ma-sadeghi]

Is there a way to switch between the two at runtime?

[carstenbauer]

While it is easy to switch from OpenBLAS to MKL (using MKL), there currently isn't a simple way to switch back to OpenBLAS (see #58 (comment)). Similarly, there is no "switch back and forth" function yet. But I think this would be really useful!

[ViralBShah]

With libblastrampoline, it should be possible to do so in julia 1.7.

[carstenbauer]

Yeah, in #58 we already agreed that it would be great to have a built-in function for switching back to the default BLAS. It's on my todo list (but not very high priority). So if someone want to beat me to implementing it, feel free to do so :)

[cossio]

I would also be interested in having this. Related: https://discourse.julialang.org/t/from-mkl-back-to-openblas/37722

[cossio]

In case someone runs into this, one can switch at runtime using LinearAlgebra.__init__() and MKL.__init__():

julia> using LinearAlgebra

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libopenblas64_.so

julia> using MKL

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libmkl_rt.so

julia> LinearAlgebra.__init__()

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libopenblas64_.so

julia> MKL.__init__()
4770

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libmkl_rt.so

Credits to @giordano for suggesting this to me, and for also mentioning that this is not the "official way" to do this.

[carstenbauer]

Why close? If this is fixed in LinearAlgebra, can you link the corresponding issue/commit?

[ViralBShah]

The suggestion that @cossio made above should work, right?

[carstenbauer]

Well, it works, yes, but it isn't a proper solution IMO. It relies on internals (i.e. isn't part of documented API) and thus might stop working at any point (also might have side effects).

[ViralBShah]

Ok, we can keep it open until we have a robust solution.

[ctkelley]

Amazing. It works once, but does not seem to be reversible.

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.1 (2023-06-07)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libopenblas64_.0.3.21.dylib

julia> using AppleAccelerate

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
├ [ LP64] Accelerate
└ [ILP64] Accelerate

julia> LinearAlgebra.__init__()

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libopenblas64_.0.3.21.dylib

julia> using AppleAccelerate

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libopenblas64_.0.3.21.dylib

[ctkelley]

My bad. Using

AppleAccelerate.__init__()

instead of using AppleAccelerate does the job.

[ViralBShah]

We just need a nice API in base Julia to reset back to openblas.

[ctkelley]

Here's a hack job that is working for me.

module BlasMania

using AppleAccelerate
using LinearAlgebra
using LinearAlgebra.BLAS

export OPblas
export AAblas
export WhichBlas

function OPblas(verbose=false)
   LinearAlgebra.__init__()
   verbose && WhichBlas()
   return 
end

function AAblas(verbose=false)
   AppleAccelerate.__init__()
   verbose && WhichBlas()
   return
end

function WhichBlas()
   display(BLAS.get_config())
end

function __init__()
   LinearAlgebra.__init__()
end

end

[ViralBShah]

Would it be better in general to have an API for switching BLAS providers, rather than doing something by default when the packages like MKL.jl or AppleAccelerate.jl are loaded?

[ctkelley]

Would it be better in general to have an API for switching BLAS providers, rather than doing something by default when the packages like MKL.jl or AppleAccelerate.jl are loaded?

I'd like that, especially if it goes into base.

[giordano]

I mean, there's already:

BLAS.lbt_forward(OpenBLAS_jll.libopenblas_path; clear=true)

Everybody's using LinearAlgebra.__init__() because I once was asked to do that and I was lazy and suggested to run the __init__ of LinearAlgebra, but the line above is what LinearAlgebra.__init__() does.

[ctkelley]

Here's a version that avoids the XX.__init__() calls. It is working for me.

@giordano @ViralBShah have I left anything out and/or broken anything?

module BlasMania

using LinearAlgebra, LinearAlgebra.BLAS, OpenBLAS_jll

export OPblas, AAblas, WhichBlas

global const libacc = "/System/Library/Frameworks/Accelerate.framework/Accelerate"

function OPblas(verbose=false)
   BLAS.lbt_forward(OpenBLAS_jll.libopenblas_path; clear=true)
   verbose && WhichBlas()
   return 
end

function AAblas(verbose=false)
# Load LP64 interface first
   BLAS.lbt_forward(libacc; suffix_hint="\x1a\$NEWLAPACK", verbose=false, clear=true)
# Load ILP64 interface next
   BLAS.lbt_forward(libacc; suffix_hint="\x1a\$NEWLAPACK\$ILP64", verbose=false)
   verbose && WhichBlas()
   return
end

function WhichBlas()
   display(BLAS.get_config())
end

function __init__()
   BLAS.lbt_forward(OpenBLAS_jll.libopenblas_path; clear=true)
end

end

[ViralBShah]

Yes - I think it should be possible to have a single package that allows you to pick a BLAS of your choice. Is it a good idea to combine all the BLAS into a common package and then lazily pick the BLAS implementation at runtime (including installing it and such using some of the new fangled package extensions?)

[ViralBShah]

Also note, now there is a ReferenceBLAS_jll package in Yggdrasil.

[carstenbauer]

Yes - I think it should be possible to have a single package that allows you to pick a BLAS of your choice. Is it a good idea to combine all the BLAS into a common package and then lazily pick the BLAS implementation at runtime (including installing it and such using some of the new fangled package extensions?)

Doesn't sound like a bad idea. I might give it a try if I can find the time for it.

Also note, now there is a ReferenceBLAS_jll package in Yggdrasil.

What does it do?

[imciner2]

Also note, now there is a ReferenceBLAS_jll package in Yggdrasil.

What does it do?

It is just the netlib reference BLAS, so you can think of it as just the for-loop versions of BLAS without any manual or architecture-specific optimizations (outside of what the compiler will do naturally).

[carstenbauer]

Ah, I see, thanks!

[ctkelley]

How do I get to the reference BLAS. I tried and got this ...

julia> BLAS.lbt_forward(ReferenceBLAS_jll)
ERROR: UndefVarError: `ReferenceBLAS_jll` not defined
Stacktrace:
 [1] top-level scope
   @ REPL[4]:1

[imciner2]

You need to pass the library handle from the JLL package to the forward function, so it should be:

julia> using ReferenceBLAS_jll

julia> BLAS.lbt_forward(ReferenceBLAS_jll.libblas)

[ViralBShah]

Also note, now there is a ReferenceBLAS_jll package in Yggdrasil.

What does it do?

The main idea is to have it for certain use cases where OpenBLAS and others use too much memory, where performance is not critical but memory/simplicity is.

[ctkelley]

I'm still confused. When I try to load the reference BLAS, OpenBLAS won't go away and any timings I try to do are the same as with OpenBLAS. When I load AppleAccelerate, OpenBLAS vanishes.

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.0-beta1 (2023-07-25)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using ReferenceBLAS_jll

julia> BLAS.lbt_forward(ReferenceBLAS_jll.libblas)
150

julia> display(BLAS.get_config())
LinearAlgebra.BLAS.LBTConfig
Libraries: 
├ [ILP64] libopenblas64_.dylib
└ [ILP64] libblas.3.11.0.dylib

[giordano]

#90 (comment) you're missing the clear=true keyword argument.

[ViralBShah]

julia> BLAS.lbt_forward(ReferenceBLAS_jll.libblas_path; suffix_hint="64_", clear=true, verbose=true)
Generating forwards to /Users/viral/.julia/artifacts/2cb0087642eb574a4cbfa6602fe5acdcd46c354d/lib/libblas.3.11.0.dylib (clear: 1, verbose: 1, suffix_hint: '64_')
 -> Autodetected symbol suffix "64_"
 -> Autodetected interface ILP64 (64-bit)
 -> Autodetected normal complex return style
 -> Autodetected gfortran calling convention
 -> CBLAS not found
Processed 4949 symbols; forwarded 150 symbols with 64-bit interface and mangling to a suffix of "64_"
150

julia> BLAS.lbt_forward(LAPACK_jll.liblapack_path; suffix_hint="64_", verbose=true)
Generating forwards to /Users/viral/.julia/artifacts/e799279503a7d85631b23b4b9eedb22149019094/lib/liblapack.3.11.0.dylib (clear: 0, verbose: 1, suffix_hint: '64_')
 -> Autodetected symbol suffix "64_"
 -> Autodetected interface ILP64 (64-bit)
 -> Autodetected normal complex return style
 -> Autodetected gfortran calling convention
 -> CBLAS detected
Processed 4949 symbols; forwarded 1668 symbols with 64-bit interface and mangling to a suffix of "64_"
1668

With refernce BLAS and LAPACK:

julia> @time lu(randn(1000,1000));
  0.176198 seconds (5 allocations: 15.267 MiB)

With default (openblas):

julia> @time lu(randn(1000,1000));
  0.024504 seconds (5 allocations: 15.267 MiB)

[ctkelley]

Thanks @giordano. That did it.

[ctkelley]

Nope.

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.0-beta1 (2023-07-25)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using ReferenceBLAS_jll

julia> BLAS.lbt_forward(ReferenceBLAS_jll.libblas; suffix_hint="64_", clear=true, verbose=true)
Generating forwards to /Users/ctk/.julia/artifacts/2e4869ba781dc4cd63120d1b68f6be4914f55573/lib/libblas.3.11.0.dylib (clear: 1, verbose: 1, suffix_hint: '64_')
 -> Autodetected symbol suffix "64_"
 -> Autodetected interface ILP64 (64-bit)
 -> Autodetected normal complex return style
 -> Autodetected gfortran calling convention
Processed 4949 symbols; forwarded 150 symbols with 64-bit interface and mangling to a suffix of "64_"
150

julia> A=rand(2,2);

julia> lu(A)
Error: no BLAS/LAPACK library loaded!
ERROR: SingularException(2)

[ViralBShah]

That's because you didn't forward LAPACK. Once you clear, all BLAS and LAPACK symbols don't point to anything. At that point, you are only forwarding BLAS, but not LAPACK. See my example above.

[ctkelley]

Somewhere the using LAPACK_jll got lost in your example. Seems to work now. Thanks.

[carstenbauer]

Yes - I think it should be possible to have a single package that allows you to pick a BLAS of your choice. Is it a good idea to combine all the BLAS into a common package and then lazily pick the BLAS implementation at runtime (including installing it and such using some of the new fangled package extensions?)

Doesn't sound like a bad idea. I might give it a try if I can find the time for it.

Before I give this a go, I would like to hear the opinion of the (main) authors of some of the other LBT-forward packages, specifically, @giordano (FujitsuBLAS), @staticfloat (MKL), @simonbyrne (AppleAccelerate).

Personally I like the idea of a single, unified "BLAS Switcher" package. After all, each of the backend-specific packages has essentially only a few lines of code these days. The central question for me is whether we should

• A) put the LBT forwarding logic for all the backends (not too many lines of code) directly into this package and thus effectively deprecating the individual backend packages
• B) only agree on an interface, e.g., every backend package should provide a, say, "lbt_forward_me" function, such that the "BLAS Switcher" package can just use this interface for LBT forwarding a specific backend.

For perspective, FujitsuBLAS.jl and BLISBLAS.jl currently have around 20 lines of code each. MKL.jl has around 50. AppleAccelerate.jl is bigger (because it also provides array-oriented functions) but if we only count the LBT forwarding part we are definitely <100 as well. Overall, we're thus talking about <≈ 200 lines of code.

What do you guys think?

PS: I think that just BLAS.jl might be a good name for this overarching package.

[ViralBShah]

I think a single BLAS.jl is a great idea. Of course it would also do LAPACK. Although one might argue that using our stock LAPACK is better except in some cases like MKL.

But only makes sense if the binary artifacts can be installed based on user preferences. Ideally we would then use the Preferences mechanism to pick the BLAS. I think the individual packages can simply be deprecated. They mostly exist as a way to install the JLL and do the forwarding.