LLM Gateway

The LLM Gateway (powered by LiteLLM) routes AI requests to OpenAI, Anthropic, and Gemini with rate limiting, cost tracking, and a unified API interface.

The LLM Gateway provides a single OpenAI-compatible endpoint for all AI providers. Configure your provider API keys in the gateway, then route all your tool integrations through one URL.

Access

URL: https://llm.<env-name>.ops.aithroyz.com

LiteLLM UI: https://llm.<env-name>.ops.aithroyz.com/ui

Using the unified API

# OpenAI-compatible endpoint — works with any OpenAI SDK
curl https://llm.<env-name>.ops.aithroyz.com/v1/chat/completions \
  -H "Authorization: Bearer <your-litellm-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

# To use Claude instead, just change the model:
# "model": "claude-3-5-sonnet-20241022"

API KeysRead article →MCP ToolsRead article →