Failed to run on Mac mini M4 with Metal

[ShiSheng233]

Hi there,

I built whisper.cpp by following the Quick start guide on my Mac mini M4 and tested it with the tiny.en model.
The building process was successful, but when executing the whisper-cli program, it throws thousands of lines of errors related to ggml_metal_init and kernel.

I've attached the entire log of whisper-cli for this issue. Please check it out.

run.stderr.log
run.stdout.log

---

CMake log:

> cmake -B build -DWHISPER_SDL2=ON

CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.


-- The C compiler identification is AppleClang 16.0.0.16000026
-- The CXX compiler identification is AppleClang 16.0.0.16000026
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /opt/homebrew/bin/git (found version "2.48.1")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: arm64
-- Including CPU backend
-- Accelerate framework found
-- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES) 
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) 
-- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND) 
CMake Warning at ggml/src/ggml-cpu/CMakeLists.txt:53 (message):
  OpenMP not found
Call Stack (most recent call first):
  ggml/src/CMakeLists.txt:318 (ggml_add_cpu_backend_variant_impl)


-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- ARM -mcpu not found, -mcpu=native will be used
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod - Success
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm - Success
-- Performing Test GGML_MACHINE_SUPPORTS_sve
-- Performing Test GGML_MACHINE_SUPPORTS_sve - Failed
-- ARM feature DOTPROD enabled
-- ARM feature MATMUL_INT8 enabled
-- ARM feature FMA enabled
-- ARM feature FP16_VECTOR_ARITHMETIC enabled
-- Adding CPU backend variant ggml-cpu: -mcpu=native+dotprod+i8mm+nosve 
-- Looking for dgemm_
-- Looking for dgemm_ - found
-- Found BLAS: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk/System/Library/Frameworks/Accelerate.framework
-- BLAS found, Libraries: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk/System/Library/Frameworks/Accelerate.framework
-- BLAS found, Includes: 
-- Including BLAS backend
-- Metal framework found
-- The ASM compiler identification is AppleClang
-- Found assembler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Including METAL backend
-- SDL2_INCLUDE_DIRS = /opt/homebrew/Cellar/sdl2/2.30.12/include;/opt/homebrew/Cellar/sdl2/2.30.12/include/SDL2
-- SDL2_LIBRARIES    = SDL2::SDL2
-- Configuring done (3.3s)
-- Generating done (0.1s)
-- Build files have been written to: /Volumes/SSD 2T/Development/whispers/whisper.cpp/build

> cmake --build build --config Release

[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  2%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  6%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  7%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[  8%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[  9%] Linking CXX shared library libggml-base.dylib
[  9%] Built target ggml-base
[ 11%] Generate assembly for embedded Metal library
Embedding Metal library
[ 12%] Building C object ggml/src/ggml-metal/CMakeFiles/ggml-metal.dir/ggml-metal.m.o
[ 13%] Building ASM object ggml/src/ggml-metal/CMakeFiles/ggml-metal.dir/__/__/__/autogenerated/ggml-metal-embed.s.o
[ 14%] Linking C shared library libggml-metal.dylib
[ 14%] Built target ggml-metal
[ 16%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[ 17%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[ 18%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[ 19%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[ 20%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[ 22%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[ 23%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[ 24%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[ 25%] Linking CXX shared library libggml-cpu.dylib
[ 25%] Built target ggml-cpu
[ 27%] Building CXX object ggml/src/ggml-blas/CMakeFiles/ggml-blas.dir/ggml-blas.cpp.o
[ 28%] Linking CXX shared library libggml-blas.dylib
[ 28%] Built target ggml-blas
[ 29%] Building CXX object ggml/src/CMakeFiles/ggml.dir/ggml-backend-reg.cpp.o
[ 30%] Linking CXX shared library libggml.dylib
[ 30%] Built target ggml
[ 32%] Building CXX object src/CMakeFiles/whisper.dir/whisper.cpp.o
[ 33%] Linking CXX shared library libwhisper.dylib
[ 33%] Built target whisper
[ 34%] Building CXX object examples/CMakeFiles/common.dir/common.cpp.o
[ 35%] Building CXX object examples/CMakeFiles/common.dir/common-ggml.cpp.o
[ 37%] Building CXX object examples/CMakeFiles/common.dir/grammar-parser.cpp.o
[ 38%] Linking CXX static library libcommon.a
[ 38%] Built target common
[ 39%] Building CXX object examples/CMakeFiles/common-sdl.dir/common-sdl.cpp.o
[ 40%] Linking CXX static library libcommon-sdl.a
[ 40%] Built target common-sdl
[ 41%] Building CXX object examples/cli/CMakeFiles/whisper-cli.dir/cli.cpp.o
[ 43%] Linking CXX executable ../../bin/whisper-cli
[ 43%] Built target whisper-cli
[ 44%] Building CXX object examples/bench/CMakeFiles/whisper-bench.dir/bench.cpp.o
[ 45%] Linking CXX executable ../../bin/whisper-bench
[ 45%] Built target whisper-bench
[ 46%] Building CXX object examples/server/CMakeFiles/whisper-server.dir/server.cpp.o
[ 48%] Linking CXX executable ../../bin/whisper-server
[ 48%] Built target whisper-server
[ 49%] Building CXX object examples/quantize/CMakeFiles/quantize.dir/quantize.cpp.o
[ 50%] Linking CXX executable ../../bin/quantize
[ 50%] Built target quantize
[ 51%] Building CXX object examples/stream/CMakeFiles/whisper-stream.dir/stream.cpp.o
[ 53%] Linking CXX executable ../../bin/whisper-stream
[ 53%] Built target whisper-stream
[ 54%] Building CXX object examples/command/CMakeFiles/whisper-command.dir/command.cpp.o
[ 55%] Linking CXX executable ../../bin/whisper-command
[ 55%] Built target whisper-command
[ 56%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/talk-llama.cpp.o
[ 58%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama.cpp.o
[ 59%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-adapter.cpp.o
[ 60%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-arch.cpp.o
[ 61%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-batch.cpp.o
[ 62%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-chat.cpp.o
[ 64%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-context.cpp.o
[ 65%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-cparams.cpp.o
[ 66%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-grammar.cpp.o
[ 67%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-hparams.cpp.o
[ 69%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-impl.cpp.o
[ 70%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-kv-cache.cpp.o
[ 71%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-mmap.cpp.o
[ 72%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-model-loader.cpp.o
[ 74%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-model.cpp.o
[ 75%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-quant.cpp.o
[ 76%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-sampling.cpp.o
[ 77%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/llama-vocab.cpp.o
[ 79%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/unicode.cpp.o
[ 80%] Building CXX object examples/talk-llama/CMakeFiles/whisper-talk-llama.dir/unicode-data.cpp.o
[ 81%] Linking CXX executable ../../bin/whisper-talk-llama
[ 81%] Built target whisper-talk-llama
[ 82%] Building CXX object examples/lsp/CMakeFiles/lsp.dir/lsp.cpp.o
[ 83%] Linking CXX executable ../../bin/lsp
[ 83%] Built target lsp
[ 85%] Building CXX object examples/deprecation-warning/CMakeFiles/main.dir/deprecation-warning.cpp.o
[ 86%] Linking CXX executable ../../bin/main
[ 86%] Built target main
[ 87%] Building CXX object examples/deprecation-warning/CMakeFiles/bench.dir/deprecation-warning.cpp.o
[ 88%] Linking CXX executable ../../bin/bench
[ 88%] Built target bench
[ 90%] Building CXX object examples/deprecation-warning/CMakeFiles/stream.dir/deprecation-warning.cpp.o
[ 91%] Linking CXX executable ../../bin/stream
[ 91%] Built target stream
[ 92%] Building CXX object examples/deprecation-warning/CMakeFiles/command.dir/deprecation-warning.cpp.o
[ 93%] Linking CXX executable ../../bin/command
[ 93%] Built target command
[ 95%] Building CXX object examples/wchess/libwchess/CMakeFiles/wchess-core.dir/WChess.cpp.o
[ 96%] Building CXX object examples/wchess/libwchess/CMakeFiles/wchess-core.dir/Chessboard.cpp.o
[ 97%] Linking CXX static library libwchess-core.a
[ 97%] Built target wchess-core
[ 98%] Building CXX object examples/wchess/wchess.cmd/CMakeFiles/wchess.dir/wchess.cmd.cpp.o
[100%] Linking CXX executable ../../../bin/wchess
[100%] Built target wchess

Part of whisper-cli log:

whisper_backend_init_gpu: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M4
ggml_metal_init: picking default device: Apple M4
ggml_metal_init: using embedded metal library
ggml_metal_init: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:61:35: error: unknown type name 'block_q4_0'
void dequantize_q4_0(device const block_q4_0 * xb, short il, thread type4x4 & reg) {
                                  ^
<lines of error>

ggml_backend_metal_device_init: error: failed to allocate context
whisper_backend_init_gpu: failed to initialize Metal backend
whisper_backend_init: using BLAS backend

[nabeards]

Seeing the same issue here on an M1 Pro.

[nabeards]

Turned out for me I had the regular base model installed but was trying to use the medium CoreML model I built. They need to match (obviously)!