This chapter is testing the Llama 2 easily.
I am going to deploy web service using FastAPI and Llama 2.
Run web server
git clone https://github.com/choonho/llama_server.git
cd llama_server
pip3 install llama-cpp-python langchain
pip3 install fastapi uvicorn
Prepare Llama 2 model file
In step 2, we created Llama 2 model file, copy to "models/7B/ggml-model-q4_0.bin"
Run Server
python3 server.py
Test (Q&A)
Open the web browser, https://localhost:8000/docs
FastAPI provides easy way to test API.
Click "Try it out"
In the "Request body", ask your question.
Series
Llama 2 in Apple Silicon Bacbook (1/3)
https://dev.to/choonho/llama-2-in-apple-silicon-macbook-13-54h
Llama 2 in Apple Silicon Bacbook (2/3)
https://dev.to/choonho/llama-2-in-apple-silicon-macbook-23-2j51
Llama 2 in Apple Silicon Bacbook (3/3)
https://dev.to/choonho/llama-2-in-apple-silicon-macbook-33-3hb7