Serve models

Expose an OpenAI-compatible API. Existing clients can talk to Buraaq Mind.

Start serve

buraaq ai serve Qwen/Qwen3-8B
buraaq ai serve Qwen/Qwen3-8B --port 8000 --bind 127.0.0.1
buraaq ai serve MODEL --backend vllm --quantize 4bit --gpus 0,1
buraaq ai serve MODEL --strata auto

Defaults: --backend auto, --strata auto, bind 127.0.0.1:8000. The planner prints the plan (quant, tensor parallel, VRAM, Strata residency) before load. Quality is never silently degraded. Models larger than VRAM: Strata.

Endpoints

Path	Notes
`GET /v1/models`	Listed model id
`POST /v1/chat/completions`	Chat; `"stream": true` → SSE
`POST /v1/completions`	Completions
`POST /v1/embeddings`	Proxied if upstream supports it; else 501
`GET /health`	Liveness
`GET /metrics`	Text counters (requests, tokens approx, uptime)

Auth (optional):

$env:BURAAQ_AI_KEY = "secret"
buraaq ai serve MODEL --api-key secret

Clients send Authorization: Bearer secret.

Status and bench

buraaq ai status
buraaq ai bench MODEL

Status scrapes live /health and /metrics when serve is up. Bench records load time and approximate tokens/sec with methodology stated (not a synthetic win).

Env

Variable	Role
`BURAAQ_AI_BASE_URL`	Upstream for `--backend openai` and `std.ai`
`BURAAQ_AI_KEY`	API key for serve / clients
`BURAAQ_AI_CACHE`	Model cache root

Isolation

The facade is a small HTTP peer. Real weights run in llama.cpp / vLLM / another OpenAI-compatible process. Prefer loopback bind; put TLS at your reverse proxy.

Next: Train adapters or Deploy AI.