Providers, routing, and fallbacks

The Gateway can route chat completion requests to four production providers and supports a hierarchical fallback chain when the primary provider is unavailable.

Supported providers

A mock provider is available for local development when MOCK_MODE=true is set. The mock provider returns a pre-determined response without calling any upstream API.

Route profiles

A route profile defines a capability category and maps it to default models across providers:

When a request does not specify a route, the Gateway uses text-general.

Provider ordering

The Gateway determines which provider to call first using the following priority:

1. Request-level providerOptions.gateway.order — if set, this list takes

highest priority.

1. Cortex router selection — when the request includes a provider selected

by the router, that provider is tried first.

1. Environment default — set the ETHEN_DEFAULT_PROVIDER environment

variable to openai, anthropic, deepseek, or openai-compatible.

1. Auto-detected order — all available providers are tried in the order

openai → anthropic → deepseek → openai-compatible.

Providers that are not configured (no API key set), have an open circuit breaker, or are blocked by a project's provider allowlist are skipped.

Fallback behavior

When the primary provider fails, the Gateway falls back to the next available provider in the chain:

• Same-provider retries — the Gateway retries the primary provider up to the

configured limit if the failure is classified as retryable (transient network error, timeout, rate-limit, or server error).

• Cross-provider fallback — after exhausting retries on the primary provider,

the Gateway moves to the next provider in the ordered chain.

• Cortex fallback policy — when Cortex is active, its fallback policy can

override both retry counts and cross-provider fallback behavior.

Each provider attempt is logged with status, latency, and error details, visible in the request logs.

Circuit breaker

The Gateway tracks per-provider failures using a circuit breaker. When a provider fails repeatedly, the circuit opens and the provider is temporarily excluded from routing. A successful request closes the circuit and restores the provider to the eligible pool.

Per-request routing controls

The Gateway supports OpenAI-style providerOptions.gateway in the chat completions request body for fine-grained routing:

Example:

{
  "model": "openai/gpt-4o-mini",
  "messages": [{"role": "user", "content": "Hello"}],
  "providerOptions": {
    "gateway": {
      "order": ["anthropic", "openai"],
      "only": ["anthropic", "openai"],
      "models": {
        "anthropic": "claude-sonnet-4-20250514"
      },
      "providerTimeouts": {
        "anthropic": 30000
      }
    }
  }
}

Unknown or invalid provider IDs in the routing controls are silently dropped — they never cause a 400 error.

Provider allowlist

Projects can configure a provider allowlist to restrict which providers are eligible for use. When a project has any allowlist entries:

• Only explicitly allowed providers are considered for routing.
• Providers with no matching entry are denied by default.

When no allowlist is configured (the default), all available providers are eligible. Allowlist enforcement requires a configured Supabase storage backing. If storage is unavailable the check fails closed: provider access is denied unless GATEWAY_BYPASS_ALLOWLIST_CHECK=true is set (local development only).

Listing providers

GET /api/gateway/v1/providers
Authorization: Bearer ***

Returns metadata about all known providers including configuration status and runnable model counts:

{
  "data": [
    {
      "providerId": "openai",
      "providerName": "OpenAI",
      "providerKnown": true,
      "health": {
        "configured": true,
        "setupRequired": false
      },
      "runnableModelCount": 42
    }
  ],
  "meta": {
    "providerCount": 5,
    "knownProviderCount": 5,
    "runnableProviderCount": 2
  }
}

See also

• Gateway overview
• Authentication and API keys
• Quickstart
• Bring your own key (BYOK)