Open AI infrastructure

Control every AI request.

Route each task to the right model. See where every token goes. Set the limits before costs, failures, or provider risk become operational problems.

Deploy Arbr ↗ See the integration

One OpenAI-compatible endpoint. Self-hosted. Provider-neutral.

POST /v1/chat/completions model:auto

CONNECT

Normalize access across every provider.

100+ providers1500+ models1 endpoint

SEE

Understand every request before routing.

taskTypedifficultycost

ROUTE

Choose the right model for the task.

ruleAI policyguardrail

GOVERN

Enforce limits before execution.

budgetrate limitkill switch

SELECTED PROVIDEROpenAI

gpt-4o-mini

184mslatency

$0.0024cost

63%saved

One control layer across the models your teams already use.

ANTHROPICOPENAIGEMINIBEDROCKDEEPSEEKGROQ

The control plane

Your AI stack needs an operational layer.

Arbr sits between applications and model providers, applying the same routing, logging, budget and governance policies to every request.

Route intelligently

Match requests by task, application, workflow, department or user. Use human rules, cost guardrails or reviewable AI policies.

Explore routing ↘

See the full system

Trace cost, latency, tokens, failures, cache performance and routing decisions across providers, applications, teams and users.

View observability ↘

Govern before scale

Set budgets, rate limits, API-key attribution and application kill switches. Keep provider credentials away from client applications.

Review controls ↘

System architecture

One gateway between your apps and every model.

Point any application at a single OpenAI-compatible endpoint. Arbr classifies, routes, meters and governs each request, then delivers it back — over a fabric of 100+ providers and 1500+ models.

ClientsPoint any app at one OpenAI-compatible endpoint.

Control planeClassify, route, meter and govern every request.

DeliversResponse, analytics and recommendations back out.

ProvidersOne layer over 100+ providers and 1500+ models.

SYSTEM ARCHITECTURE LIVE

MODEL PROVIDERS

100+ providers · 1500+ models · one endpoint

Frontier labs

AnthropicOpenAIGoogle Gemini

Cloud

Amazon BedrockAzure OpenAIVertex AI

Fast · open

GroqDeepSeekMoonshotxAIMistral

Via gateway

LiteLLMNVIDIA NIM

Operational visibility

Know what your AI estate is doing.

Measure spend, performance and reliability by provider, model, application, workflow, team and user. Drill into the exact reason behind any routing decision.

Overview

Usage, cost and system health across your AI estate.

Last 30 days

Total requests

1.82M

↑ 14.2%

Total cost

$6,281

↓ 18.4%

Avg. latency

612ms

↓ 9.6%

Realised savings

$4,392

↑ 22.1%

Requests and cost

Spend by provider

$6.2ktotal spend

ApplicationModelTaskDecisionCost

support-copilotgemini-flash-litesummarisationguardrail$0.0018

sales-assistantclaude-sonnetanalysisrule$0.0321

invoice-extractgpt-4o-miniextractioncache$0.0000

Request intelligence↗

Explain every route

See which rule matched, which policy fired, whether difficulty changed the model and which fallback was taken.

Realised savings↗

Measure actual economics

Re-price served tokens at the requested model’s rate and track the savings created by each routing choice.

Provider health↗

Catch degradation early

Compare 24-hour error rates and latency across providers before reliability problems reach your applications.

Response caching↗

Skip repeated inference

Return identical responses instantly and track cache hits, token savings and cost avoided in the overview.

Integrate once

Change the base URL. Keep the stack.

Arbr is OpenAI-compatible, so your SDK, agent framework and chat UI keep working. All requests immediately gain routing, logging, authentication and budget enforcement.

✓Streaming chat completions

✓OpenAI-compatible embeddings

✓Realtime voice proxy

✓JavaScript and Python SDKs

# Keep using the OpenAI SDK
from openai import OpenAI

client = OpenAI(
    base_url="https://arbr.yourcompany.com/v1",
    api_key="ab_your_gateway_key"
)

response = client.chat.completions.create(
    model="auto",
    messages=[{
        "role": "user",
        "content": "Summarise this ticket"
    }]
)

print(response.choices[0].message.content)

// Keep using the OpenAI SDK
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://arbr.yourcompany.com/v1",
  apiKey: "ab_your_gateway_key"
});

const response = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Summarise this ticket" }]
});

console.log(response.choices[0].message.content);

curl https://arbr.yourcompany.com/v1/chat/completions \
  -H "Authorization: Bearer ab_your_gateway_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{
      "role": "user",
      "content": "Summarise this ticket"
    }]
  }'

Governance in the path

Policies that act before inference.

Cost caps, rate limits and application controls sit in the data path. They remain effective regardless of which provider or model a developer requests.

Budget enforcementAlert, downgrade or block by application, provider, team, workflow or model.

ENFORCED

Gateway API keysHashed keys, per-key request limits and automatic application attribution.

ACTIVE

Application kill switchDisable all requests from an application without deleting its configuration.

READY

⌁

Encrypted provider keysCredentials stay server-side and are never exposed to client applications.

SECURED

Control cannot be an after-the-fact dashboard.

Arbr places policy where it can change outcomes: between every application request and every model provider.

Put one operational layer across every model.

Deploy Arbr, point your existing clients at the gateway and start building a governed view of your AI estate from the first request.

View project on GitHub ↗Read deployment docs