LOCKWhat “running an LLM locally” means
You download (or attach) model weights and run inference on your CPU/GPU. Prompts stay on-box unless you deliberately export them. This is the core of private, self-hosted language AI.
Pick a model class that matches hardware
7B–14B class models are common on strong personal machines. Mid-size code models need more RAM/VRAM. Sparse multi-trillion-parameter open libraries are Lab-class: high RAM, huge NVMe, job-oriented latency — not phone chat.
Quantisation in one minute
Quantisation shrinks weights so they fit more hardware, with quality trade-offs. Catalogs should document format and floors so you are not guessing alone.
Typical DIY stack
People use tools like Ollama, llama.cpp, vLLM, or custom engines. DIY works for tinkerers. Product paths add checksums, purpose docs, licence notes, and system integration.
Curated catalog approach
S◉LOCK targets installable model plugins: `silock models list`, `info`, `install`, `enable` — with hardware and purpose docs. Large weights still live on your disk; they are not smuggled into a multi-terabyte OS ISO.
Performance honesty
Tokens per second depend on model size, quant, CPU/GPU, and memory bandwidth. Frontier sparse models under Helmsman are budgeted jobs, not fantasy ChatGPT speed claims.
Next step: Explore the Helmsman model catalog, check hardware floors, or request Founding Preview access.
FAQ
- Do I need a GPU?
- No for many smaller models. A GPU helps speed and larger contexts. Helmsman Lab work is often RAM and NVMe bound as much as GPU bound.
- Is it legal to run open models?
- Follow each model’s licence. Open weights are not a free commercial blank cheque for every use case. Read SPDX and upstream terms.