Bug: Unable to load phi3:3B(2.2GB) model on Apple M1 Pro #9049
Labels
bug-unconfirmed
critical severity
Used to report critical severity bugs in llama.cpp (e.g. Crashing, Corrupted, Dataloss)
stale
What happened?
I tried to run this command:
./llama-cli -m phi3:latest.gguf -p "I believe the meaning of life is" -n 128
and it fails to load the model with the following error:
llama_init_from_gpt_params: error: failed to create context with model 'phi3:latest.gguf'
I usually run ollama with no issues on this same machine. And I just thought to try out llama.cpp using a light weight model like Phi3 but it looks like llama.cpp is failing to allocate memory.
Note: this same commands work for llama models. e.g
llama3:8b.gguf
works fine. could it be a phi3 issue? do i need some extra configs?Laptop specs:
Apple Macbook pro with M1 Pro
Mem: 16GB
OS: Macos Sonoma 14.6
Name and Version
./llama-cli --version
version: 3590 (4b9afbb)
built with Apple clang version 15.0.0 (clang-1500.1.0.2.5) for arm64-apple-darwin23.6.0
What operating system are you seeing the problem on?
Mac
Relevant log output
The text was updated successfully, but these errors were encountered: