Skip to content

Inference Llama 2 in one file of pure JavaScript(HTML)

License

Notifications You must be signed in to change notification settings

epicure/llama2.js

 
 

Repository files navigation

llama2.js

llama2.js

A pure JavaScript port of Karpathy's llama2.c with a simple UI.

How to run

  1. Download Karpathy's Llama2 (Orig instructions) parameters pretrained on TinyStories dataset

    wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin
    wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories42M.bin
    wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories110M.bin
  2. Open run.html via a WebServer

     python -m http.server 8080
     open http://localhost:8080/run.html

Performance

Tokens/sec measurement on Apple M1

tok/s 15M 42M 110M
🐢 ~30 ~13 ~5

License

MIT

About

Inference Llama 2 in one file of pure JavaScript(HTML)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 48.6%
  • C 27.6%
  • HTML 22.2%
  • Makefile 1.6%