Ollama Quick Experiment
terminal
ollama pull llama3.1
ollama run llama3.1
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "llama3.1",
"messages": [{ "role": "user", "content": "hello" }]
}'vLLM Server Experiment
terminal
vllm serve meta-llama/Llama-3.1-8B-Instruct \
--host 0.0.0.0 \
--port 8000
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "meta-llama/Llama-3.1-8B-Instruct",
"messages": [{ "role": "user", "content": "hello" }]
}'Selection Criteria
Ollama
Good for personal development, quick model swaps, and local Mac/Linux tests.
vLLM
Good for GPU servers, OpenAI-compatible serving, and throughput comparison.
Hosted API
Good for current model quality, operational reliability, and fast product validation.
Gateway
Useful when you need base URL switching and routing policies across models.