Portkey: enterprise LLM gateway with EU hosting and self-host tier

What is Portkey?

Portkey (portkey.ai) is a proprietary LLM gateway with an additional self-host tier for enterprise customers. As of May 2026, the vendor states support for over 1,600 LLM and embedding models from 250+ providers behind an OpenAI-compatible interface. The company was founded in 2023, per its own statements closed a Series A round of around USD 15M in February 2026 (named lead investor: Lightspeed Venture Partners; please verify current status), and operates cloud regions in the US (us-east, us-west), EU (Frankfurt), and India (Mumbai).

The feature set goes beyond a classic routing gateway. Portkey combines four building blocks: first, AI gateway with model routing, fallback chains, retries, and load balancing; second, observability with prompt/response tracing, cost tracking per client/workspace, and audit trail; third, guardrails with 50+ pre-built filters (PII detection, toxicity, prompt-injection detection, jailbreak pattern detection); fourth, prompt management with versioning, A-B testing, and template library.

The architecture is hybrid. The standard tier is multi-tenant cloud (portkey.ai endpoint). The enterprise tier runs as single-tenant cloud in one of the three regions or as a self-host deployment on customer infrastructure (Kubernetes Helm chart, Docker Compose stack). Self-host is chargeable and starts in the enterprise contract – typical licence cost sits at USD 30,000-80,000/year depending on volume and support level.

For fairlane.systems mandates, Portkey is a valid alternative to LiteLLM when the client wants guardrails and prompt management from one vendor and the budget covers the enterprise licence. For pure routing and audit requirements, LiteLLM remains the cheaper choice; once 50+ guardrails, semantic caches, and a managed prompt repository are required, Portkey becomes competitive.

Why it matters for Swiss mandates

Three properties make Portkey interesting for Swiss FADP and professional-secrecy contexts. First: EU region Frankfurt. Portkey can be configured so that every request, every log entry, and every trace lands only on Frankfurt servers – no routing through US nodes, even for telemetry. That covers the revised FADP requirement on avoiding third-country transfer, provided the upstream LLMs also sit in EU regions (Azure OpenAI EU, Mistral La Plateforme, Anthropic Claude on AWS Bedrock EU).

Second: guardrails as a configuration layer. A fiduciary can activate PII masking centrally before the LLM call; the filter detects Swiss AHV numbers, IBAN, UID numbers and replaces them with placeholders before the prompt leaves the gateway. On the response, a second guardrail can detect hallucination patterns or validate output schema. All this is maintained as YAML or UI configuration – no custom development needed.

Third: audit trail in a format auditors accept. Portkey writes per request an audit entry with timestamp, virtual key, model, token count, cost, guardrail violations, and response hash. These data can be exported as CSV/JSON or replicated via API into a WORM backend (Hetzner Storage Box with append-only, AWS S3 Object Lock). For Art. 957a CO reviews, that is the right abstraction.

The Series A funding, per the vendor closed in February 2026, has also driven roadmap consolidation. Portkey states it holds the compliance certificates SOC 2 Type II and ISO 27001 (please verify status and scope directly in its Trust Center); a dedicated EU account manager for DACH customers has, per the vendor, been reachable since March 2026. That lowers the bar for procurement processes in larger Swiss companies.

How it works

Onboarding runs via the Portkey dashboard at app.portkey.ai. After account creation – for EU compliance, the Frankfurt region is chosen during onboarding – a workspace is created with members, roles (Admin, Developer, Viewer), and provider keys. Provider keys (OpenAI, Anthropic, Azure, Mistral, Google) are stored centrally; applications do not use these but virtual Portkey keys.

Application integration follows the OpenAI schema:

import openai client = openai.OpenAI( api_key="pk-...", # Portkey Virtual Key base_url="https://api.portkey.ai/v1", default_headers={"x-portkey-virtual-key": "vk-mistral-eu", "x-portkey-config": "config-id-prod"} ) resp = client.chat.completions.create(model="mistral-large-2411", messages=[...])

The x-portkey-config points to a configuration in the dashboard. This configuration defines: model fallback chain (e.g. mistral-large -> claude-opus-eu -> gpt-4o-eu), retry strategy, cache behaviour (exact or semantic), active guardrails, and output schema. Changes to the configuration take effect without code release.

The observability layer records every call with prompt, response, tokens, latency, model, cost, and guardrail result. The dashboard has built-in client and workspace filters; a per-client per-month cost report is three clicks away. Exports go as CSV, JSON, or via webhook to external systems. Data retention is configurable (7, 30, 90 days, or unlimited in the enterprise tier).

Self-host setups run via a Helm chart for Kubernetes or via Docker Compose for smaller installations. Data is stored in PostgreSQL and ClickHouse; object storage (S3-compatible) is provided for prompt archives. A pilot installation on Hetzner with 3 nodes is up in a day, a production installation with HA and backup strategy in 3-5 days.

Portkey pilot in 5 steps

1. 01Create a workspace at app.portkey.ai in the Frankfurt region, store provider keys (Mistral, Anthropic, OpenAI).
2. 02Define the configuration: fallback chain mistral-eu -> claude-eu, guardrails PII mask + toxicity, cache exact 24h, retention 90 days.
3. 03Create virtual keys per application, budget (e.g. CHF 50/month for a pilot client), model whitelist (only mistral-eu-*).
4. 04Switch applications: base_url=https://api.portkey.ai/v1, headers x-portkey-virtual-key, x-portkey-config; tests with eval set.
5. 05Compliance review: per-client cost report, guardrail report, audit trail export reviewed; plan self-host migration if needed.

When Portkey fits

First, in mid-to-large setups with compliance requirements that LiteLLM alone does not cover. When a client wants a quarterly audit review with guardrail violation reports, prompt versioning history, and per-client cost reports in one system, Portkey is the single-tool solution. Multiple LiteLLM components plus Langfuse plus a custom PII filter would be the decomposed self-build.

Second, in multi-tenant platforms that offer LLMs as a sub-feature. A SaaS platform with 200 clients and a chat feature can use Portkey workspaces to maintain per-client budgets, model whitelists, and cost reports. The workspace model is designed for multi-tenancy.

Third, in teams that take prompt management seriously. Anyone maintaining 30+ prompts in production with different versions per client and A-B tests benefits from the Portkey prompt repository. Prompts are stored there as versioned artefacts, deployed per environment (dev/staging/prod), and tested with eval sets against gold standards.

Fourth, when procurement sets certificate requirements. Larger Swiss companies often demand SOC 2 Type II and ISO 27001 as minimum requirements – Portkey states it holds these certificates (please verify current status). Pure OSS solutions without certificates fail at this hurdle.

When not to use

First, in small setups with one provider and under 10,000 requests per month. Here Portkey is oversized. A direct OpenAI or Mistral call with some PostgreSQL logging is enough – or a small LiteLLM instance on a CHF 12/month Hetzner server.

Second, when the budget does not cover the enterprise licence. The self-host tier starts at USD 30,000/year, the cloud Pro tier from USD 99/month plus token-based fees. For an SME with five productive LLM calls per day, that is not appropriate – LiteLLM self-host on Hetzner runs at CHF 240/year server cost.

Third, in pure OSS-stack requirements. Some mandates (e.g. public sector, universities) explicitly require OSS components without commercial licence. Portkey is proprietary – there is no OSS variant. In those cases LiteLLM, Helicone self-host, or APISIX are the right choice.

Fourth, when an existing Kong/APISIX/NGINX gateway is already in production. A second gateway layer is redundant and complicates debugging. Here the AI plugin of the existing gateway can be added and, if needed, Langfuse can run in parallel as a pure observability component.

Trade-offs

FAQ

Related topics

Sources

1. Portkey AI Documentation – gateway, observability, guardrails, prompts · 2026-05
2. Portkey Pricing and Enterprise Tier (Cloud and Self-Host) · 2026-05
3. Portkey Series A announcement (vendor blog) – stated ~USD 15M led by Lightspeed; verify current · 2026-02
4. Portkey Trust Center (vendor self-attestation) – SOC 2 Type II, ISO 27001, EU region; verify current · 2026-04