upgrade to 1.5.0 #24

jwijffels · 2023-11-15T21:53:51Z

Stable release is now 1.5.0 and available at https://github.com/ggerganov/whisper.cpp/releases/tag/v1.5.0

shikokuchuo · 2023-11-15T22:44:22Z

Would be great to run this with the new CUDA support!

jwijffels · 2024-01-25T22:22:12Z

Work on this is in branch upgrade-v1.5.4 at https://github.com/bnosac/audio.whisper/tree/upgrade-v1.5.4
Continuous integration tests #25 which run on CPU machines already seem to work.
Regarding CUDA. I don't have a CUDA machine currently but hopefully within some months.
Some work needs to be done on the Makevars and on making sure the downloading of the models point to the right url's.

jwijffels · 2024-01-27T19:00:18Z

I've upgraded the R package to use whisper.cpp 1.5.4. A looot of backend changes.

@shikokuchuo
@rpodcast
@jmgirard

Feel free to test it out.
It should now be possible to user large-v1, large-v2, large-v3. I did some small checks that quantized models also worked, which it did.

For the CUDA support. I'll plan to look at it once I get my hands on a machine with CUDA on.
I only have access to Windows and Linux machines and the unit tests using github actions which run several platforms but it would be good if someone with a Mac could test this. Especially the compiler settings regarding the Accelerate framework.

shikokuchuo · 2024-01-27T20:17:32Z

Yep tested briefly - the quantised models quite a bit faster. Do you know what it would take to enable CUDA? I can try testing it out if there are details somewhere. Or can the package link to system libs (that I can then build with CUDA support)?

jmgirard · 2024-01-27T20:40:24Z

I can test on Mac and windows.

jwijffels · 2024-01-27T21:01:50Z

Yep tested briefly - the quantised models quite a bit faster.

I also saw that.

Do you know what it would take to enable CUDA? I can try testing it out if there are details somewhere. Or can the package link to system libs (that I can then build with CUDA support)?

I'll move this question to #27

jmgirard · 2024-01-27T21:04:07Z

Installation with default CFLAGS works on MacOS (M2 macbook).

Similarly with Sys.setenv(WHISPER_ACCELERATE = "1").

However, I get some warnings with the recommended CFLAGS:

Sys.setenv(WHISPER_CFLAGS = "-mavx -mavx2 -mfma -mf16c")
#> clang: warning: argument unused during compilation: '-mavx' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mavx2' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mfma' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mf16c' [-Wunused-command-line-argument]

jwijffels · 2024-01-27T21:04:29Z

I can test on Mac and windows.

Do you have the Mac Accelerate installed? That would help in testing.
And do you have Metal on Mac?
And CoreML on Mac?

jmgirard · 2024-01-27T21:16:56Z

I just installed accelerate via pip install accelerate and re-ran:

Sys.setenv(WHISPER_ACCELERATE = "1")
remotes::install_github("bnosac/audio.whisper", ref = "0.3", force = TRUE)

Seems to work but generates the following warnings:

whisper_cpp/ggml.c:9828:17: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
                cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans,
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
whisper_cpp/ggml.c:10238:9: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
        cblas_sgemm(CblasRowMajor, transposeA, CblasNoTrans, m, n, k, 1.0, a, lda, b, n, 0.0, c, n);
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
2 warnings generated.

Let me know if there are other things you want me to try on Mac.

jwijffels · 2024-01-27T21:24:56Z

Seems to work but generates the following warnings:

Looks indeed ok. Does that speedup transcription?

jmgirard · 2024-01-27T21:25:46Z

My macbook's system information says it supports Metal 3, but I'm not sure if I need to install anything to enable it.

I just installed CoreML via pip install coremltools. Do you need me to install any third-party packages?

jmgirard · 2024-01-27T21:28:22Z

Looks indeed ok. Does that speedup transcription?

What do you recommend for systematically testing speed? Formal benchmarking or something like tictoc? Also, would the whisper model and/or duration of the audio file matter for getting a good estimate?

jwijffels · 2024-01-27T21:32:54Z

Just get a longer audio file en see how long it takes if you have the package installed with WHISPER_ACCELERATE and without - make sure you remove the environment variable Sys.unsetenv("WHISPER_ACCELERATE") or start a new session.

library(av)
download.file(url = "https://www.ubu.com/media/sound/dec_francis/Dec-Francis-E_rant1.mp3", 
              destfile = "rant1.mp3", mode = "wb")
av_audio_convert("rant1.mp3", output = "output.wav", format = "wav", sample_rate = 16000)
library(audio.whisper)
model <- whisper("medium")
trans <- predict(model, newdata = "output.wav", language = "en", n_threads = 1)
trans$timing

jwijffels · 2024-01-27T21:35:48Z

My macbook's system information says it supports Metal 3, but I'm not sure if I need to install anything to enable it.

I just installed CoreML via pip install coremltools. Do you need me to install any third-party packages?

I need to make changes in our Makevars file to enable Metal and CoreML. This will be for later.

jmgirard · 2024-01-27T22:48:21Z

16.65 min with accelerate vs. 47.8 min without

jwijffels · 2024-01-28T06:49:40Z

Great so that accelerate option works on Mac. I will get back to this later for these coreml/metal, once I made the changes in the Makevars to enable these.

jwijffels · 2024-01-29T08:54:01Z

Installation with default CFLAGS works on MacOS (M2 macbook).

Similarly with Sys.setenv(WHISPER_ACCELERATE = "1").

However, I get some warnings with the recommended CFLAGS:

Sys.setenv(WHISPER_CFLAGS = "-mavx -mavx2 -mfma -mf16c")
#> clang: warning: argument unused during compilation: '-mavx' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mavx2' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mfma' [-Wunused-command-line-argument]
#> clang: warning: argument unused during compilation: '-mf16c' [-Wunused-command-line-argument]

Theses -mavx -mavx2 -mfma -mf16c are non-Mac settings.

jmgirard · 2024-01-29T14:40:13Z

Theses -mavx -mavx2 -mfma -mf16c are non-Mac settings.

Gotcha. I was confused because the README says:

"Common settings to set for WHISPER_CFLAGS on Mac/Linux/Windows are -mavx -mavx2 -mfma -mf16c and extra possible flags -msse3 and mssse3"

jwijffels · 2024-01-30T08:30:42Z

I've enabled the use of Mac Metal (I think - I'm not sure - haven't got a Mac myself).
Would be great if you could test with this and see if that speeds up by doing this transcription for example: #24 (comment)

Sys.setenv(WHISPER_ACCELERATE = "1")
Sys.setenv(WHISPER_METAL = "1")
remotes::install_github("bnosac/audio.whisper", force = TRUE)

jmgirard · 2024-01-30T14:51:48Z

Error when trying to build for METAL

── R CMD build ──────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T/Rtmpd2uqRK/remotes11bb064364aec/bnosac-audio.whisper-06e569c/DESCRIPTION’ ...
─  preparing ‘audio.whisper’:
✔  checking DESCRIPTION meta-information
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘audio.whisper_0.3.1.tar.gz’
   
Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘audio.whisper’ ...
** using staged installation
** libs
using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.1.0.2.5)’
using C++11
using SDK: ‘MacOSX14.2.sdk’
I whisper.cpp build info: 
I UNAME_S:  Darwin
I UNAME_P:  arm
I UNAME_M:  arm64
I PKG_CFLAGS:   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_CPPFLAGS: -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_LIBS:   -framework Accelerate -framework Foundation -framework Metal -framework MetalKit

clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:9828:17: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
                cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans,
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
whisper_cpp/ggml.c:10238:9: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
        cblas_sgemm(CblasRowMajor, transposeA, CblasNoTrans, m, n, k, 1.0, a, lda, b, n, 0.0, c, n);
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
2 warnings generated.
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c rcpp_whisper.cpp -o rcpp_whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c RcppExports.cpp -o RcppExports.o
make: *** No rule to make target `+', needed by `audio.whisper.so'.  Stop.
ERROR: compilation failed for package ‘audio.whisper’
* removing ‘/opt/homebrew/lib/R/4.3/site-library/audio.whisper’
* restoring previous ‘/opt/homebrew/lib/R/4.3/site-library/audio.whisper’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T//Rtmpd2uqRK/file11bb02eb10497/audio.whisper_0.3.1.tar.gz’ had non-zero exit status

jwijffels · 2024-01-30T17:07:07Z

Right, sorry about your time, probably this metal is fixed with this commit 32758b8
If it doesn't, I'll probably have to look for spinning up some Metal in AWS. These metal optimisations are new to me, I hope it is as simple at that commit but it might not be.

jmgirard · 2024-01-30T17:15:48Z

Not a big deal to try this on my end.

Different error

── R CMD build ──────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T/Rtmp0xWYAY/remotes12e63549897e2/bnosac-audio.whisper-32758b8/DESCRIPTION’ ...
─  preparing ‘audio.whisper’:
✔  checking DESCRIPTION meta-information
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘audio.whisper_0.3.1.tar.gz’
   
Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘audio.whisper’ ...
** using staged installation
** libs
using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.1.0.2.5)’
using C++11
using SDK: ‘MacOSX14.2.sdk’
I whisper.cpp build info: 
I UNAME_S:  Darwin
I UNAME_P:  arm
I UNAME_M:  arm64
I PKG_CFLAGS:   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_CPPFLAGS: -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread
I PKG_LIBS:   -framework Accelerate -framework Foundation -framework Metal -framework MetalKit

clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-quants.c -o whisper_cpp/ggml-quants.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-backend.c -o whisper_cpp/ggml-backend.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml-alloc.c -o whisper_cpp/ggml-alloc.o
clang -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include   -DGGML_USE_ACCELERATE -DGGML_USE_METAL -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -fPIC  -g -O2  -c whisper_cpp/ggml.c -o whisper_cpp/ggml.o
whisper_cpp/ggml.c:9828:17: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
                cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans,
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
whisper_cpp/ggml.c:10238:9: warning: 'cblas_sgemm' is deprecated: first deprecated in macOS 13.3 - An updated CBLAS interface supporting ILP64 is available.  Please compile with -DACCELERATE_NEW_LAPACK to access the new headers and -DACCELERATE_LAPACK_ILP64 for ILP64 support. [-Wdeprecated-declarations]
        cblas_sgemm(CblasRowMajor, transposeA, CblasNoTrans, m, n, k, 1.0, a, lda, b, n, 0.0, c, n);
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/vecLib.framework/Headers/cblas.h:610:6: note: 'cblas_sgemm' has been explicitly marked deprecated here
void cblas_sgemm(const enum CBLAS_ORDER __Order,
     ^
2 warnings generated.
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/whisper.cpp -o whisper_cpp/whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common-ggml.cpp -o whisper_cpp/common-ggml.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c whisper_cpp/common.cpp -o whisper_cpp/common.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c rcpp_whisper.cpp -o rcpp_whisper.o
clang++ -std=gnu++11 -I"/opt/homebrew/Cellar/r/4.3.2/lib/R/include" -DNDEBUG -DGGML_USE_METAL -DSTRICT_R_HEADERS -I./dr_libs -I./whisper_cpp  -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -pthread -I'/opt/homebrew/lib/R/4.3/site-library/Rcpp/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/xz/include -I/opt/homebrew/include    -fPIC  -g -O2  -c RcppExports.cpp -o RcppExports.o
make: *** No rule to make target `ggml-metal.o', needed by `audio.whisper.so'.  Stop.
ERROR: compilation failed for package ‘audio.whisper’
* removing ‘/opt/homebrew/lib/R/4.3/site-library/audio.whisper’
* restoring previous ‘/opt/homebrew/lib/R/4.3/site-library/audio.whisper’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/kr/tx86v16n5bx_djz_z2cpvfkc0000gq/T//Rtmp0xWYAY/file12e634c64562c/audio.whisper_0.3.1.tar.gz’ had non-zero exit status

jwijffels · 2024-01-31T08:19:35Z

Thanks for your time, I'll move this Metal integration to issue #34

jwijffels · 2024-02-02T10:44:54Z

My macbook's system information says it supports Metal 3, but I'm not sure if I need to install anything to enable it.

I just installed CoreML via pip install coremltools. Do you need me to install any third-party packages?

Now that we have Accelerate/Metal working, I also enables coreml (#35)

jwijffels · 2024-03-08T08:38:16Z

@shikokuchuo Feel free to test auto the CUDA integration - it's enabled on the master branch (see #27).

shikokuchuo · 2024-03-08T12:37:57Z

Fantastic - I will take it for a spin when I get some time!

shikokuchuo · 2024-03-08T22:42:30Z

Wow the sample file translation was instantaneous. May be able to actually make use of this now - I've put it on my list for closer inspection. Thanks for letting me know about the update.

jwijffels closed this as completed Jan 27, 2024

jwijffels mentioned this issue Jan 31, 2024

Integrate Metal #34

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

upgrade to 1.5.0 #24

upgrade to 1.5.0 #24

jwijffels commented Nov 15, 2023

shikokuchuo commented Nov 15, 2023

jwijffels commented Jan 25, 2024 •

edited

Loading

jwijffels commented Jan 27, 2024

shikokuchuo commented Jan 27, 2024

jmgirard commented Jan 27, 2024

jwijffels commented Jan 27, 2024

jmgirard commented Jan 27, 2024

jwijffels commented Jan 27, 2024

jmgirard commented Jan 27, 2024 •

edited

Loading

jwijffels commented Jan 27, 2024

jmgirard commented Jan 27, 2024

jmgirard commented Jan 27, 2024

jwijffels commented Jan 27, 2024 •

edited

Loading

jwijffels commented Jan 27, 2024 •

edited

Loading

jmgirard commented Jan 27, 2024

jwijffels commented Jan 28, 2024 •

edited

Loading

jwijffels commented Jan 29, 2024

jmgirard commented Jan 29, 2024

jwijffels commented Jan 30, 2024

jmgirard commented Jan 30, 2024

jwijffels commented Jan 30, 2024 •

edited

Loading

jmgirard commented Jan 30, 2024 •

edited

Loading

jwijffels commented Jan 31, 2024

jwijffels commented Feb 2, 2024

jwijffels commented Mar 8, 2024

shikokuchuo commented Mar 8, 2024

shikokuchuo commented Mar 8, 2024

upgrade to 1.5.0 #24

upgrade to 1.5.0 #24

Comments

jwijffels commented Nov 15, 2023

shikokuchuo commented Nov 15, 2023

jwijffels commented Jan 25, 2024 • edited Loading

jwijffels commented Jan 27, 2024

shikokuchuo commented Jan 27, 2024

jmgirard commented Jan 27, 2024

jwijffels commented Jan 27, 2024

jmgirard commented Jan 27, 2024

jwijffels commented Jan 27, 2024

jmgirard commented Jan 27, 2024 • edited Loading

jwijffels commented Jan 27, 2024

jmgirard commented Jan 27, 2024

jmgirard commented Jan 27, 2024

jwijffels commented Jan 27, 2024 • edited Loading

jwijffels commented Jan 27, 2024 • edited Loading

jmgirard commented Jan 27, 2024

jwijffels commented Jan 28, 2024 • edited Loading

jwijffels commented Jan 29, 2024

jmgirard commented Jan 29, 2024

jwijffels commented Jan 30, 2024

jmgirard commented Jan 30, 2024

jwijffels commented Jan 30, 2024 • edited Loading

jmgirard commented Jan 30, 2024 • edited Loading

jwijffels commented Jan 31, 2024

jwijffels commented Feb 2, 2024

jwijffels commented Mar 8, 2024

shikokuchuo commented Mar 8, 2024

shikokuchuo commented Mar 8, 2024

jwijffels commented Jan 25, 2024 •

edited

Loading

jmgirard commented Jan 27, 2024 •

edited

Loading

jwijffels commented Jan 27, 2024 •

edited

Loading

jwijffels commented Jan 27, 2024 •

edited

Loading

jwijffels commented Jan 28, 2024 •

edited

Loading

jwijffels commented Jan 30, 2024 •

edited

Loading

jmgirard commented Jan 30, 2024 •

edited

Loading