Preview
Local versus hosted models
How local model runtimes (Ollama, LM Studio) compare with the Ethen Gateway — privacy, latency, capability, and setup trade-offs.
Ethen supports both local and hosted model execution. Which one you use depends on your privacy requirements, hardware, latency tolerance, and model quality needs.
Comparison table
| Factor | Hosted (Gateway) | Local |
|---|---|---|
| Model selection | 100+ provider-hosted models via the Gateway catalog. | Open-weight models only (Llama, Mistral, Phi-3, Qwen, etc.). |
| Quality ceiling | Access to frontier models (Claude, GPT-4o, Gemini). | Smaller, less capable than frontier models. |
| Latency | Variable — depends on provider and plan. | Deterministic — depends on local hardware. |
| Privacy | Request data leaves your machine. | All inference stays on-device. |
| Setup | API key only. | Requires Ollama or LM Studio installation. |
| Cost | Per-token billing from providers. | Free (compute cost only). |
| Offline | Requires internet. | Fully offline-capable. |
When to use hosted (Gateway)
Use the Gateway when you:
- Need frontier model quality (Claude, GPT-4o, Gemini).
- Want multi-provider fallback for reliability.
- Prefer zero local setup.
- Are building an API-facing service.
When to use local
Use local runtimes when you:
- Must keep all data on-device (sensitive or regulated workloads).
- Need deterministic latency in a controlled environment.
- Work offline frequently.
- Are prototyping without cloud provider costs.
Managing both
Both local and hosted models appear in the same Console interface. You can switch between them freely per request. The model picker shows:
- Gateway models with a cloud icon.
- Local models from your running Ollama or LM Studio instance.
Local models you have loaded are available in the Console chat and the Code workspace alongside hosted models.
Getting started with local models
- Install Ollama or LM Studio.
- Pull a model (e.g.
ollama pull llama3.2). - The model appears automatically in the Console model picker.
See the Gateway Quickstart for hosted setup.